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During 1974, materials developed by the Australian 
Science Education Projects (ASEP) became available in final published 
form. The publication of these materials was- an especially important 
^undertaking since ASEP was the first national curriculuun project, in 
any subjiect area, to be developed in Austraij-ia. This report presents 
research into four^spect-s of ASEP evaluation . All studies were 
t^^conducted at the seventh grade level ahd^invblyed students in 
Victo ria high schools. Questionnaires iwere use*d in the first study ^ 
O^ljapter 2) to investigate teachers I understanding of ASEP philosoi:)hy 
and 4ihe impact of ASEP materials on tlieir ideas about science 
teaching. In the second stucjy (chapter' 3), aibatteiry of learning 
outcome measures were administered to ASEP/nbn-ASEP students at the 
beginning and end of the school year Xp examine the ef f ectiveness-of 
ASEP/non-ASEP materials in promoting learning changes. The impact of 
^ the learning environment- as a focu^ for\ curriculum evaluation and 
reseai^ph related to theflearning environment^^in. .ASEP classrooms was 
investigated in the, thirds-study (chapter\ 4) . The f ourtifc study C> 
(chapter 5) made a comparison between classroom crimate percept^Qns 
of ASEP/non-ASEP students and looked at tiie relationship betweeiv^ 
those perceptions and student learningl Objectives, methodology, 
results, "and conclusions ^re provided for each of the studies. 
(Background itfformat idn , origin/nature of ASEP, contemporary scene in 
^0lence , education , and first/second generation curriculum projects 
are^ discussed in tlie^lntroductgry section of the report Appendixes 
present 'results of two 1974 surveys oi science teachers' knowledge 
and use of ASEP, sample items from scales Tneasuring learning 
outcomes, ^nd examplelS of studeftt activity charts.! (JN) ^ 
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FOREWORD 



The CDC has taken over responsibility foV maintaining the 
Australian Science Education Project which, from 1969 to 1974, 
developed materials for a new approach to«science in grades 
seven^to ten of secondarjls^hools. ASEP was the first program of 
curriculum materials development by States carried out on an 
Australia*wide basis. It was supported in full by funlls-made 
available by the Australian and all^tate Departments, of Education. 

ASEP materials were first introduced within schdqls in late . 
1974 and, by the "beginning of 1976, were being widely used. 
They have since become a major resou^rce for secondary science 
teachers. , ^ . • 

A considerable number of studies, some^supported'b^ the 
tpC, have been made of the impact^n and utilisation by 
schools of ASEP. These include The Impact of the Australian Science 
'Edu/^tim Project in Schools^ a study carried out for the Australian 
Council of Educational Research by'Mr John Owen, and aReviejv 
of Research on the Australian Science Education Project by Dr Barry 
Fteer. ^ 

Thp volume aow published was prepared by Dr Barry 
Eraser and Mr Jeff Northfield. It reports a study, carried out at 
Monash University under the overall direction of Professor Peter 
Fensham, of the impact of ASEP and the problems it hss posed 
♦ for evaluators and of the particular approaches adopted by the 
writers in meeting these problems. The study investigates:^ 
. ■ . ■ . 

• the iihpact of ASEP on science teachers 

the effectiveness of ASEP promoting pupil learning 

• the learning environnient as a f6cus for evaluating ASEP 

• learning environment variables 

Thus the report is of jnterest both for the information it 
provides on the impact of AgEP on AustraliaA schools and for its 
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consideration of meithodological issues i^evaluation, during the 
dissemination phase of a niajor natiorial , project _ 
The CDC is pleased to make this report available as part ot 
its continuing ]^esponsibility Tor ASEP and as a contribution fo 
curriculum evaJuation'studies. 

u;, ■ - ■ , " - • ■ 

MALCOLM„SKILBECK ■ ' ' 

DII^CTOR- '* \ y 
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PREFACE 

The introduction of A^EP has provided new challenges and 
problems for evaluation of the program. This repor^t attempts to 
outline the problems and describe our approaches to dealing' 
with the challenges that ASEP posed for evaluators. In this report 
these approaches to evaluation. have tended to be emphasised 
.more than the actual results obtained. Further details ^bout the 
results can be found in articles published by the authors (See 
References). The analysis of some of the information is con^imiing 
and further publications will complete the picture that emerges 
from this study. > 

The authors are nappy to provide further details of the study 
to people interested in these aresS of ASEP evaluation. 

To conduct a study of the type described in this report 
requires the encouragement and support of many people and 
•organisations. In our case we are indebted-to the following: 

^ • The CDC for haying the confidence in our proposal 

• Peter Fensham for his r willing support and sponsorship 
throughout the study 

m The many staff of the education faculty at Monash University 
who provided assistance''in niany aspects of the project with a 
special mention to Lindsay Mackay 

• . Elaine Scott, a very patient typist 1 „ 

• • Mr T.J. Ford, Director of Secondary Education in the Victorian 

Education Department who willingly granted approval to 
approach the teachers involved in the study 

• Finally, thii Grade? science teachers who not only volunteered 
to take part in various aspects of the study but in many cases 
displayed an enthusiasm and interest |ar exceeding our 
expectations. We hope that this report is to some extent 
worthy of the teacher support we retjeived. 



Barry Fraser JefT Northfield 



BACKGROUND 



During 1074, materials developed by the -Australian Science 
Education Project (ASEP) first became available in final published 
form. The publication of these materials must be considered an 
important event in the history of Australian education since 
ASEP was the first natiorial curriculum project in any subject area 
in this country. ^ ^ " 

As well as being Australia's first national curriculum venture, 
ASEP has .^provided a stimulus and a focus for numerous and 
varied research endeavours/* In particular, a number of studies 
have involved the important task of evaluating the effectilveness 
of ASEP materials. The evaluation of ASEP, however, has 
presented a challenge to the educational researcher Decause 
differences between ASEP and conventional-materials have 
rendered some of the standard reisearch methods inappropriate. 

The purpose of this report is to describe research into four 
aspects of the evaluation^of ASEP. Th^se four aspeas,^ Which are * 
presented in Jater chapters, are: ^ 

• the impact of ASEP on science teachers; 

• the effectiveness of ASEP in promoting pupil learning; 

• the learning environment as a fobus for evaluating ASEP; 

• further researdi involving learning environment variables. 



Areas discussed in this chapter include the contemporary scepe 
in science education and the nature of ASEP materials and 
philosophy. . J 

CPNTEMPORARY SCIENCE EDUCATION ^ . 

The last two decades have seeif changes and developments in 
science education which haveibeen described as 'yevolution- 
ary- ' This increased activity hasiinvolved the establishment of a 
large number of'cumcu/um projects' and several commentators 
have found it useful to distinguish between yirii generattomnA 
second generation ^TO]ccts. • , . j 

Some important changes in philosophy have been claimed 
for second generatio'n projects in recent years when compared to 
-first generation projects. The next section will look at sortie of 
these distinctiions. _ ' ^ ■ ^ 

First and Second tJ^eration Curriculum Projects 

Manywriters have argued that the launching of Sputnik J by 
the Russians in ^957 gave major impetus ka science cumculuri[i , 
reform movement in the United §tates.' llie years immediately 
following the launaung saw the appearaike of the first nauonal 

• curriculum projeci?^. as PSSC and %SCS, These projects 
and others like them had two common features which st^emmed 

• from a central cohcem for improved^ sHentific manpower. 
• First, it wa« thought important to concekrate on reforming 

and ubdatinfcontent in kchieientific disciplihe taught in school. 
Second, this desire fo/upd/£e science contenUed to thegnlistm^t 
of professional scientistsih the development of curriculum matenals 
In fact, in 1 963, there were ten Nobel Laure&t^s actively inyolved 
in school science p/rojects compared to^ none in the previous 
menty-five years.' / \ \ \'' ' .:' 

r The enlistment of professional scientists ih first generation 
projects was successful in producing ?ccurate\and up to date 
materials but there were a number of important shortcomings 



/Which led to a change in philosophy for the seconci generation 
projects. The shift in eipphasis is marked by the rationale and 
philosophy underlying Harvard Project Physic**^ Other examples 
of second generation science curriculum projects are Intennediate 

Science Curriculum Study (ISCS) for usq in the junio(^ high 
sth^ol, and Elementary School Science (ESS) for us/ in the 

^primary or elementary school.*" ^ ' . ^ 

An important feature of first generatic^i projects \yas t\\^\T 
t-eliance^n up to date content from ^scientific disciplines. 
However they» neglected two other important features which 
were recognised in s<;fond generati(!)n projects; first, the integration 
ofdiscipiinesy.cmiphasisxng connecrfb|ns between different scieroi/Tc 
disciplines and going beyond traditional scientific disciplines • 
to- anthropology, psychology, etc."; second, the emphasis « 
on humanistic aspects of science including social, historical, 
cultural and ethical considerations..'^ Another feature of first 
generation projeas was the attempt to make curriculum materials 
'teacher-prooP. That lis, there was a belief that the best way to 
convey science to stulderits was via curriculum packages whic^^ 
could be passed down from developer to student by way of the - 
teacher, but without . the imprint of the teacher's ideas, style or 
personality. In contrai\t, second generation project materials 
attempted to provide a degree of teacher^choice,^^ First generation 
projects also tendjfdirf^ neglect the pupil as Ian implortant ' 
determinant of the nature of curriculunni. materials. In second 
generation projects, however, the importance Of the pupil was 
acknowjedged in two ways/ First, they recognised individual 
differences in pupil abilities and interest, and provided for a certain 
degree of pupil choice.'^ Sec'ond, while first generation projects 
seemed to assume that materials based on th^ content of scierke 
would automatically prove interesting toVstuden^,'* second 
generation project^, were deliberately designed to promote pupil' 
interest in science.'*' In fact, one second generation project, 
naniely Harvard Project Physics, has the s^tated aim of fostering , 
sufficient interefs^i; in physics to reverse trends of decliningphysics 
enrolments at the senior high school level." Although the above 
discussion has applied particularly to projects developed in the 



. United States, numerous important national science curriculum 
project (e:g. Nuffield) emerged in the United Kingdom dunng:.^ 
the same period: Unlike th^merican or British scene, howweip^ 
Australia: has no history of national cumculuni projecp. ^^^^ 

: ^ the firsfnational curriailum project to be ; developed^ my 
Australia. Its origins, philosophy and materials, and m particuto, ^ 
its relationship to second generation projects^ wiU now be ; 

-V- examined. ' ' ■ : ' " ■y^v.i 

the Australian Science Education Project (ASEP) , . ■ 

• /Thejunior Secondary Sdence Project aSSE),thefore-m^ 
" ASEP, Sis^set lip ,in' 1966 joindy by, the Science Standing 
i f- Commiffee of the Yictbrian Universities and Schools Examinauon 
• • Board ^and the Australian Council for Educational Research. 
Following favourable reports jfrom teachers usingj SSP materials 
during 1 966-68, an approach-was made to the Federal tSoverri- 
ment for funds to instigate a more comprehensive science 
curriculum project. Between October 1969 an« March 1974, 
:$1 .2 raiUion was made available, with $750,000 provided by the^ 
'Federal Government and the remainder provided by all six; 
Australian States. Work on JSSP was tenninated and attention 
mmed to a new project, named die Australian Science EducaL^on 
Projed,' which aimed to produce matterials suitable for Grad^ 7^| 
10 sderice teaching in ail States,"* , 1 

■ feachfers, 6ffidals from die departments of educanon in^ 
^ch St^t^ scientists, and science educators from colleges an4, 
universities attended'a guidelines conference in January I97(K^ 
These guideUnes provided die starting point from which_ the. 
aims and philosophy of ASEP evolved. Between the time of the 
conference aiid early 1974, MEP produced atotal^of forty-one 
units for student use,^each occupying approximately a month s 
teaching time, as well as six service booklets ^d audio-visual 
, materials. Protot^es of most units were written insa first trial 
form^which was tried oupn Victoria, and tiien in a second tnal 
form Which was tried outnationaUy''' ASEP placed emphasis^on 
integration of disdplines, humanistic asperts of saence, teacher 
■ ' ■ : ' : ■ ■ ' A ' ' . 
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gaibice; 1^ individual di^ pupU interest in science. 

■^A^ been ■ mM^/to^ integrate tracl^onal science 

social scien^e"'^is€ii^^ 
iising die environment as an integradhg tl^n^ 
some attehdon to humani aspects of science by stressing the 
influence of. science on liuman welfare^andf oniaesdietic and 
ecological aspects of die environment," and by afcknowledging 
the importance of the history of science." The. brganisadon of 
ASEP materials into a large number of units awhich are relatively 
independent of each otlier allocs teachers to choSq^ 
to us^ if any,' and in what sequence. Units are ^ structured 
according to Piagetian levels of development^ particular Attention is 
paid to readability of materials, students can choose between 
options within each unit and each pupil can^proceed through 
materials at his or her own rate.'* Fpstejing pupil intjerest in 
science is a statc^i aim of ASEP," and was a criterion in selecting 
die topics for ^inclusion in ASEP/units." Therefore, ASEP 
possesses, JO a reasonabte j^gree,' all five cHaraaeristics of 
second generation cun^icuIurnWojects, \ \ ' 

■ ' ^ / y J ■ ■ ■/ • 

EVALUATION OF ASEP — FROBLJEMS AND ' > 
GUIDELINES 

The field of curriculum evaluation is a relatively new one. In fact,- 
Popham" has . identified the 1967 essays of Stake and Scriven as 
die real staning point of the curriculum evaluation field. Since 
then, the rate of publication of articles on curriculum evaluation 
hais increased dramatically.^* . . J f 

A feature of recent theoretical writings on curriculum 
evaluation is the large variety of methodologies and criteria 
which have been suggested for evaluating curricular effectiveness. 
This can be seen in the evaluation models proposed by Metfessel 
and Michael," Sanders and Cunningham,^ and Fr^er.^^ Alterna- 
tive approaches' to the evaluation of a curriculum could involve 
die collection of opinions of teachers using the curriculum, 
measurement of pupil attainment, content analysis of curriculum 



^ ^tenak tliemselves, or the study of the learning environment in ^ 
-^assrolns iTsingthe curriculum. A curriculum evaluator must . 
therefore cTedde on the icope of the -study and the cntena to i 
ludee curriculum effectiveness. • . r u V^^^^^^ 

ThisreportlooksatfouraspectsofASEP. Each of these four 
studies was carried out during 1974 - the first-year ASEP v 
materials were available. Th^ were conducted at the seventh 
grade level and involved^upiis in Victonan high schools , , A ; 
Lgle request was sent to a nAmber of schooh eariy^ .197f . 
asking for co-operation in a series of studies called the Mon^h 
University Science Evaluation ^'roject (IvtuSEP). The first smdy, • 
reported in Chapter 2, set out^o evaluate the impact of ASEP on 
science teachers. Mackay^' has contended that.the classroom 
teacher is an important potential source of information for use jin 
curriculum ev^uation, and a number of studies related to ASEP 
have involved surveys of teacher opinions," A quesuonnaire 
' investigated teachers' urAerstanding of AS^P^philosophy and^ 
the impact of ASEP irtateriMs on teachers' ideas about saence , . 

^^^"^The second study (Chapter 3) lookld at the effectiveness of 
ASEP in promoting pupU learning. A battery of learning outcorne 
measure; was administered to ASEPand non-ASEP pupdsat the; 
beginning and end of the school year to find out the effi^ave- • • 

■ heJs'of ASEP and non-ASEP materials in promotingOgarning 
change'^^iuring the year." * „ r f^ir 

The impact of the learning environment A? a tocus tor 
curriculum evaluatioh~aiidjeseaxch_r^^ 
-environment in ASEP classrooms was investigated in the third , 
study (Chapter 4) . The fourth study (Chapter 5) made a ?ompan- 
son between the 'classroom climate perceptions of ASEP and 

■ .non-ASEP pupils and also looked at the re^tionship between 
r V) those perceptions and pupil learning. - 

SUMMARY ^ ■ 

This chapt^as discussed the background of the present 

research. The contemporary scene nn science educauon was 



and first and second generation curriculum projects 
?|i^ were distinguished. The origin and nature of ASEP was considered 
V and it that ASEPs pkUosophy and materials were^ 

I . consistent with. the orientations/of second generation projects. 
5: The evaluation of ASEP was di/cussed and a brie^verview was 
' given of the four studies into the evaluation of ASEP described in 
the present report 
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' • As part of its prograxn for the development of its materials 
ASEPdeyeJoped a model teacher education prograQi: ASEP 
rec(^^ teacher as the key to the l^an^ 

V i^SEp.:>^ can be used in a: variety of learning 

y ^[^Itu^^ the project saw the roles of teachers using the 

materials.^ from those of teachers in conventional 

^j^^classrooms'.* -, ■ --'-h- ■ 

■/■VV* •"■/ ■ ••; ■■ '" - ' ■ ■ ' 

■p^ clearly recognised the importance of the , 

teache^ theife:i;ijOT development progrsun although th^ 

% teachen^d^ was l^ter Hmited by finance and became 

; 2^ in the ultimate diffusion and implementation 

^ pf the ASEP' matierials? 1- \ 

^^; : T]hie 'first sm^ on the impact of ASEP on^e 

^>/^qence stfeskeci 
|^i|^4^mppi^^ throughout its existence and this is " 

f fe^^ teachen^The 
•rfinal rhaterials could not be regarded as * teacher- proof yet 
teachers v^^^ to conisider a certain philbisophy of'science 



I ■ education,* a unique approach *\ organising the content and . ^ 

' teaching kpproaches which departed from the normal approaches u 

' to science in schools.' The trend for teachers to make important i?^ 
? curricuium decisions' now became important with the inu-oduc- 

; . tibii of a wide varie ty of units eKh<with s(^^ 

- ' a number of curriculum adoption and irriplementationMies|p 
have described the way teachers 'translate' cuixicuhim^atenals 

' into their dassrboins and the ass<k 
which hav^ developed w^^^^^ 

i and the appropriate teacher educatibri" are based: orifa^de^ 

I . understanding oPthe teacher in the context of the 

I Thirdiy.^curiiculum evaluation-information from tea<^^^ 

' is based on actud experience.'" Teachers albnef caii respond 'to 

" the tyt^s of infbrmation they received prior tpT the intrpductiori^ 

' of^ curriculum and the deficiendes of the^rriculum as 

" operating in a classroom:" Hurd" provides a description of a . 

study of the effects of BSfcS materials oh biological educa^on.H 

distinguishes between an impaa study and a curriculum evalua- 
tion study although both types of study o^^eriap in many respects 
Many of the qiiestions Kurd asks in his study can be answered 
only by gath^ng data from teacher^ ' ; 

To sutai up, teaches provide^ ai> important source ot y 
-evaluatiori ^ta because of the nature of the ASEP curripilunr g 
and their artual experience of the difiusipn and implementauoirg? 
of the curriculum. It should alsb be noted that man^^^^ 
'alternative' approaches to evaluation referred to by Stenhouse 
invol ve gathering information^in a variety of ways from a variety 

; DESIGN OF THE S-mJDY 

In February, 1974, a questionnaire was sent t4 Grade 7 , science 
teachers at Victoriar/ cb-educational State high schools. The 
questionnaire was direrted to the prindpal of each school With a 
request that if be/passed on to the appropriate teachers^vMore 
than two hundred schools were contacted in this way and by the| 
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end of March replies had been received from 15.1 teachers in 1 21 
schools. In October 19 Zi . the initial resporjdents were asked to 
complete a second questionnaire and by midi-N'ovember 108 
replies JhaH been received. The following table sets out the 
information being sought from the^cience teachers and Appendix 
1 presents the responses obtained: ■ ; " 

INFORMATION SOUGHT FROM THE TWO 
•QUESTIONNAIRES TO GRADE 7 SCIENCE TEACHERS 



~ — ' — ' ~r " 

Infofnution 


Fkst 
Quest ioniiatre 

C^k UmmI* I til A ■ 


. Secon4 . 
Questionnaire 
Oci'Nov 1974 


BACKGROUND INFORMATION 

Schuul. Name. Teaching bxp eric nee. Quali Heat ions 


^. 

/ 




bslimatc of knuwicdge of ASBP 






Suurccs uf Informaiiun about ASEP 






FORM I SCIENCE 1974 

Science ai single subject ur integrated with , 
^oiher subjects. 


. } 
/ 




Curriculum materials used in 1974 






'Anticipaicd use of ASBP materials 






• Met hud uf making curriculum decisions 






Present Ideas about Form 1 Science 


■ J 




^/^^ Present knoyf ledge of>\SEP ^ 






;V;^ Informal ion about new cuVricula 






; Opinions abuut ASEP materials 




y 


Opiniuns abuui Furm 1 science teaching 


y 


■ ^ y. ^ 


Detailed use uf ASEP materials in 1974 

(inc. way (he unit was used and evaluation uf unit) 






' Problem are^s associated with use of ASEP 







THE GRADE 7 SCIENCE TEACHER SURVEY v f 

The two quesnonnaires Were^designed to the way in 

which ASEP materials were accepted and used by one group p^^^^^^^^^^^ 
\^ctorian science teachers. Some of the results have been made ? ; 
available 4n two journal articles'' and this section of the report ^J 
will describe some of the more.imjjfcrtant results. 

Experience of Grade 7 Science Teachers: ^ 

The following table siets out the experience of the teachers in the ' 
group responding to the first questionnaire. . 

TEACHING EXPERIENCE OF INITIAL RESPONDENTS 



Yunof 
Teaching 


FInt 
year 


1-3 
yean 


4-6 
years 


.7-10 
ye*i» 


More than 
10 yean , 

J . 


Per cen\ 
of 

sample 
(N=I51) 


t 

17 . 


34 


0 19 


18 


13^ 



One in six of these Grade 7 teachers was a first year teacher and^| 
51 per cent of the respondents had less than fouryears teqtching J 
experience. Up to 83 per cent of them could have had in- service 
teacher education and up to 5 1 per cent would havebeen likely to^ ^ 
have done a pre-service course during the ASEP development 

Anticipated Use of ASEP in the First Year' 

Before the materials were generally available, 79 per cent of the 
respondents were confident that they had at least suffiqen^ 
knowledge to consider and use ASEP materials.'' The majority of 
schools in this sample" treated science as a separate subject in 



1974 despite a trend to integrate subjeas in the junior yeara of 
^secondary schools/' Approximately seventy per cent jaL^e 
teachers intended to' use at least one unit of ASEP in fiie first 
yeai^° and in the majority bf cases Grade 7 science curriculum 
decisions had been made by the group of science StafF.^' 

^he Extent to Which AjSEP Klaterials Weije Used During 

■ V . ■■ • ■, ■ , I ■ 

By the end qf 1974, ASEP materials had been used by seventy per 
cent of respondents to the second questionnairis,^'^^ although not 
without some problems." 

Information gathered from the 108 respondents to the 
second questionnaire allowed the teachers to be placed into 
ree groups according to the use made of ASEP materials in- 
1974. 



./Sei 
/th 



Group 1 — Intensive users: j 

Twenty- six teachers who had used more than two units of 
ASEP and ASEP mateials had been used for more thair- 
one term (fourteen weeks). 

Group 2 — Moderate u 




Twenty- nine teachers who fyad used ASEP materials to a 

lesser extent than Gr 

•' ' * 

> Group 3 - Non users:.^ 

. Thirty- three teachers who had made no use of ASEP 
materials cfuring 1974. 

The remaining thirty teachers were exclbt3e3"^M)m the analysis 
because they ha^Kyplunteered for another part \)f the'^esearch' 
which required teaaj|rs*o use ASEP or alternative materials, but 
not both during the year/^ 



RELATIONSHIPS BETWEEN TEACHERS' 
ATTRIBUTES AND ADOPTION QF ASEP 

There needs to be aceitain amount of caution in the inteirpretation of 
the relationship between the attributes of the teachers^aiid Ae 
adoption of ASEP materials. Intensive use or non use^ot A&tr 
materials could be associated with many factors which may or 
may not be varijfeles included in thf study. 



Detertnining Relationships Between Initial Attributes 
and Final Use of ASEP / 



The dia^am below illustrates the initial attributes of the teachers 
which were exanJined to identify variables associated yith the 
use of ASEP materials. The literature associated with the adop^, .: 
tion of new curriclila" had presented each ofthese attributes as / 
significant factors. ^ 7 . 

VARIABLES ASSOCIATED WITH USE OF ASEP I^TERIALS 



Typ« of Infonmtlon 

Rr- 



Rtquiftd 



Ptfciption 




fMchcf Pmonil 
htrictcrbttcs 



j pefCiptlon of ASEP 



of Sci«nc< 



S|u«i in AdoP<ton* 



43% 



Awirefios 

NON USERS 

Evaluitioil ) 



JNlreNSlVE Adoplton 
'USERS - I 



(ImptafiMntttion) 



• Aft«f H«»eIock(l97q) 

/ 



SOURCES OF INFORMATION ABOUT ASEP 
. • (February — March, 1974) ' 









I 


Mean score 




■ ... ^ ' 




For teachers 




for those 






receiving . 
information. 




receiving 
infonrtation 


1 vahic 

inlemive. 




% Receiving 


in this way 


linpact. 


u dib 


users (lU) 




in^nhatidn 


% very . 


VSlHt* 


. - .«yt * 


vsno 


Sourer 


in thb way 


useful 


(Rank) 


(R*nk) 


«Mra(NU)t 


Journal Articles 


77 


17 


0.13(4) 


2.1(6) 


2.34»(IU) 


ASEP Units (Trial form) 




51 


0.43(1) 


2.5(3)* 


.1.05 


Teacher Education 












Materials 


i 57 


26 . 


0.15(3) 


2.1(4) 


1.40 


Newsletters, 


60 


15 


0.0<)(9) 


2.0(7) 


2.70MU) 


Newspapers 


■42 


2 


^Ol(IQ) 


1.3(10) 


1.55 


In-service Courses ^ 


50 


.66 


0.33(2) 


2.6(2). ; 


0.41 4 


Trials Teacher 


14 


86 


0.12(6) 


2.9(1) 


1.59 , 


Informal [>iscussion 












within the School 


70 


13 


0.09(8) 


■ 1-3(9) . 


1.69 


Infdrmal discu«ltm-^~^ 








1.9(8") 




outside the School 




16 


0.10(7) 


2.09 S{IU) 


Pre-service Courses 


30 


33 


0.13(5) 


.2.1(5) 


0.68 



♦ Impact vakfe. An attempt to compare the various sources of information 
by taking in to account extent of dissemination (Column 1) and efiectivenesa 
(Column 2). (Proportion receiving information x propornon very.useful « 
* Impact value'.) v 

t ^core based on 3 ^ very useful to 0 » no use. 

X Test foi^ difference between means qf independent sar ^es. (Intensive 
users, v.s npn- users). p0.05. Intensive users rated the sour »s more useful 
in eachj'caise. ' : ' ' ; 

'^>nd< .05 Intensive users rated the source as more useful in each (jase. 



Sources of Information Abput ASEP 

Earlier it was noted that 7-t^per cent of the respondents to the fim ; 
questionnaire said they knew at least enough ^bout the ASEP 
materials to consider and use them in their classro6ms. How did 
teachers obtain this information about ASEP? 

The above table sets out the answer to this as obtained 
through the questionnaire.^^ ' ^ 



It is worth noting th^it: ^ ^ 

• The number of science teachers who had seen ASEP trial ' 
niatenals was surprising, as the ASEP projea prpduced /; 
lirhitea quantities of trial materials: When extent of >• 

/ dissemination (column !) add usefubfiess{colurnn 2) 
taken into a[ccount this source of information appears to ^ 
have had the most impact (column 3) with ithese teaqhers. ; 

• Fifty per cent of teachers had been involved in some forim ^ 
of in-service course (earlier, afigtire of 83 percent of tliej 
re^ondent^ as being the maximum tairget i 

^ --population for this type of activity). ;! 

• Thirty^nine per cent of teachers had rej:eived ii^ 

a pre-service level although up to 51 per cent of teachers | 
could have been involved in a pre-seryice course whil^ i 
the ASEP development was occurring. \^ ■ ; 

• ASEP units appear to have had most overall impact as a 
source of infoririatibn, with in-;senace^^ 

' most important source of information (column 3). - ^ 

• Partidpatibn as a trials teacher proved to be very iisefiil . ; 
for those teachers, involved, v«th in^service courses and ■ 
the ASEP units also rated as usfefimo very useful overall 

. (column'^4).^ / / ' . ^; 

' • In coluWS the ratings of subsj^quent intensive use^^ 

non-iisefs are compared. For all sources of information 
except 'pre-service courses' intensive ilsers; rated tlie 
source of information more highly than non-users. This 
may indicate that teachers who were thinking of using the 
materials were more carefully examining all possible 
sources of information. In three cases the differences in ^ 
• ratings of usefulness reached statistical significance. The; 
^ teachers who Subsequently made intensive use of the 

ASEP materials rated journal articles, newsletters and 
disoission outside the school as more useful sources of 

inforimation than non-users. , 



Types of Inform Required by Teachers 



Two descriptions of case studies related to t^e introduction of 
new curricula (Gross >^ fl/^^ and Tom'^') draw attention to the 
praictical cprtcems teachers have wTheri they consider using new 
materials. Clearly, tieacher education should be concerned more 
with practical problems than is oftein the case. The introduction 
of ASEP materials at the beginning of the 1974 school yeay gave 
an opportunity to see what types . of information were most 
important for teachers; Founeen statements were written to 
represent some of the possibly types of information which could 
be provided for teachers in teacher education associated with 
ASEP introduction. They could be grouped into four general 
areas: , . 

» ■ ■ ' 

• 'Aims, and philosophy of the program. ^ 

Practical issues of a general nature — issues tliat would be 
likely to be associated with the introduction of any new 
curriculum. * 

J • Practical issues specific to the ASEP materials. 

• Opinions -and information resulting from use of the 
materials. 

In the next table, the^fourteen statements are set out in the four 
groupings stated above.. Teachers were asked to respond to each 
statement by responding on a scale from extremely important 
(scored 3) p no importance (scored 0). The mean value and rank 
are set out for the total group of respondents and tlie groups of 
teachers^^wjjo subsequendy became intensive users and non- 
users. The follSwing points appear to be worth considering 

• For the total group, the first six statements ranked by 
teachers include the three statements related to the aims 
and philosophy of ASEP and also suggest that interest 
response of pupils and likely organisational problems 
and costs are regarded as important | 

. « ■ 



• The practical issues specific ASEP >vere not rated very ^ ^ 
highly overall yet it is worth noting that the intensive 

. users are tending to rate these as more important 
(difference between means significaunit at five per cenr^ 
le^tel in the case of two of the sutements). l>vo repom*' ^ 
discuss several ojF the pfactical problems asjsodated witji 
ASEP use which were laxgelyunantiapated before iti(^^ 
■ introduced.^*^ , ■ , , '.y /'■■^•^■yi'-y^lr-j-o^^-:^^^^ 

• There is a tendency for non-users to rate the practic^ 
issues of a general nature as more important When > 
cpmparedwith intensiveusers. The concern formethods - ^ 
of assessing pupil achievement was rated significaridy^^ 
higher by later non-users as compared to later intensive j 
users. / ^ 

Teachers' Opinions of ASEP and Fonfiv^l Science 

In both the first and last questionnaires teachers were asked their - 
opinipnsofASEP and Form 1 sdence. The initial opinions of the^^ 
intensive users and non-users of ASEP are set out below. A test 
fqr difference between means of independent samples Avas used 
to determine whether there were sigriificant (Mfferehces behyc« 
the two groups of teachers before the materials were available. 

The intensive users of ASEP seemed to regard the matenals 
as 'more formal', 'clearer* 'and 'better^ thaii the non-user g^ 
of teachers. Intensive users alsotended to regard Form X science 
as 'more interesting', 'more pleasant^, ^more importamf an^ 
'dearer* than non-usera The dif5prent views of Form 1 science 
appear to be just as important a distinguishing feature between 
the two groups as theViews held about ASEP materials. 

HOW TEACHERS SAW ASEP AND FORM 1 SGIENC E 

Herron** describes a theoretical model to explain why teachers 
may oi^ may not adopt a new curriculum. He regards die 



TYPE OF INFORMATION REQ,UIRED BY TEACHERS 





Total 


Intenth^e 




tteit 






uaen 


Non«Uaen 


difference 


Type of Information 


Mcan'(IUiik) 


Mcii^dUnk) 


Mcan1([iUnk) 


between^ 


(N-I5I) 


(N"151) 


(N-33) 


"means 


Aims. Philotoptiy Of The Prognra 










Rjtiomie and Dhilovinhv nf ACPP 




2,58(2») 


2,03(7 J4) 




Theory of learning on which the 










course^ is based 






2.03(7yiV 




Aims and objectives of the ^ 








materials * 


2.41(1) 


2 73( 1 ) 


o 44/1 ) 




Practical Imm - GcncraJ Nature 











Costs ofmaterialsjind equipment 


2.26(4) 


2J8(4) 


2.42(2) 




Knowledge of new subject matter 


1.71(11) 


1.68(12) 


^8(9) 




Ukely planning and organisational 










problems • 


2.28(3) 


2.19(6) • 


2.28(3^) 




Methods or assessing pupil 










achievement after using the 










materials 


1.80(10) 


1.77(11) 


2.09(5) 


•V 


Methods of determining the 










initial ability of pupils 


1.62(13) 


1.50(14) 


1.75(11) 




PMJcal laucs (Related to ASEP) 










Ways of integrating ASEP with 










other subjects , 


1.85(8) 


2.08(7). 


1.67(I2W) 




Ways of organising group work 


1.70(12) 


1.88(10) 


1.34(14) 




The teaching role being 










suggested ^ 


1.79(9) 


1.96(9) 


1.78(10) 




OpinJonsand Information 










from Vhth . 










Test results of pupils using 
trial materials *• 


1.44(14) 


1.54(13) 


"^.59(12^) 




Opinions of teachers who have 






- used materials 


1.96(7) 


2.00(8) 


2.06(6) 




1 Interest response of pupils 










who have used the materials 

1 


2.29(2) 


2.27(5) 

\ 


2.28(3W) 





f Score based on range from no importance 
= 0 to extremely important = 3. 



PERCEPTIONS OF ASEP 



not 




Interestlns 
Cheap 

Strange \ 
Fomul 
Complex 
Inferior 
'Conventional 
' Ordered 
Clear 
Poor 



PERCEPnONS OF SCIENCE 




Interesting 

Ordered 

Formal 

Pleasant 

DuU 

Meaninglen 
Relaxed 
Important 
Clear 



— Intensive users * p<0.05 ^nrwii 
Ncwusers t test difference between meant independent p^awi 

P^O.Ol samples 



/ 

/ 
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teachei's \aew of the subject and the teacher's view of the new 
\curriculum as very important variables, Herron's paper develops 
the concept of congruence (the extent to which the views of the 
subject and the new curriculum correspond). The congruence 
and accuracy of the views then form the basis of Herron's model. 
Twenty statements were devised to represent some o^ the 
/features of science education related to the ASEP approach to 
/ fmd out^e views of the teachers. They were asked to reply to 
each of the twenty statements, each preceded by^he introduction 

— 'A Form 1 science course should . . .' and were dien asked to 
reply to the same twenty statements preceded by the introduction 

— 'At Form 1 level ASEP materials are designed to \ . 



The responses of the 151 teachers to the first questionnaire 
were factor analysed and two factors were found to account for 65 
per cent of the variance in both the analysis of the view of ASEP 
and the view of Forni 1 science. 

Factor l: Jen statements y/ere found to emphasise this factor 
when die replies to both statements were analysed." It showed an 
emphasis on classroom' organisation invjcilving the pupil, and was 
described as a classroom organisation dimension of ASEP and/or 
Form 1 science. The replies to each of the ten statements were 
added (scored 2, 1 or 0) the higher score representing the pupil 
centred view of each teacher with respect to ASEP or Form 1 

science. / ' - 

■ ■ ' / ■ ' ' 

Factor 2: Five statements emphasisin| this factor in both 
statements were analysed. It represented the degree of curriculum 
structure required" and was described as a curriculum materials 
organisation dimension of ASEP and/or Form 1 science. The scoring 
for the five statements was completed as for Factor 1 (above), the 
higher score representing a need for structuring the content of 
the ASEP program or Form 1 science.^ 

An estimate of congruence was obtained by comparing the 
response made to each statement for Form 1 science with the 
response made to the corresponding statement for ASEP. An 
exact correspondence was scared 1 ^and lack of correspondence 
between responses 0. The accwrary of the teacher's perceptions of 
ASEP was estimated by comparing each teacher's response with 
the resjJonse agreed upon by ten 'experts' each witii a sound 
knowledge of the ASEP development and materials. For eleven 
out of the twenty statements ninety per cent of the 'experts' had 
shown agreement with a response. Thfe accuracy score for each 
teacher was calculated "as the nurhber. of statements on which^ 
there was agreement with the 'experts'. 

The table on page 24 sets out some preliminary results of the 
analysis of teachers' perceptions of ASEP and Form 1 science for 
intensive users and non-wsers of the materials. From this table it 
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■ , ■ ■ . ■' . . ■ ■ , . " ' . , ' ' 
can be seen that ihiensiye users differed from non-users in the 
first year in that they tended to have: 

J a more 'pupil centred' view of A^EP materials and Form. 
■-: 1 science; ■ ■ *,^. 7 

• a greater degree of congruence between views of_ Form 1 
science and views of ASEP; , 
' • a more accurate view of ASEP when views were compared- 
with the views x)f people very familiar with the ASEP 
development The two groups of teachers did not differ 
* in the way curriculum materials should be organised. ^ 



TEACHER PERCEPTIONS OF ASEP AND FORM 1 SCIENCE 





tatauivc. . 
Vm 
N«26 


Non* 
Uaen 

N-33 


lvalue 


ChOTocca OiiwiHtjoa Dkncuioa 

(Range p<mible 0-20 High score » 
Increued pupil ccntiediriew) 

' View of ASEP 
ViewofScknce 


- " ' '-k 
18.62 

18.08 


16.70 
15.61 


3J5** . 
3.45** 


^ CnnicyluiiiMatciitliOrpniaitkm 
DtooMioii 

(Range possible 0*10. Hi«h score - 
increased structure 

View of ASEP ^ ' 
ViewofScknce 


5.69 
6 J8 


5.85 
.- 6J6 


0.28 NS " 
0.03 NS^ 


Cbofiubicc of ASEP and Fonn 1 
ScfcaceViewa 

(Rangepossibk^O-lS) 




6.85 


3.20** ' 


Accurtcy of ASEP viewi 
. (Range possible 0-11) 


9.92, . 


8.28 


3.19** 



/ I I tes t for difference between means of independent samples. 
' 0.01 
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Concluding Comments 

. An attempt has been made to describe science teachers before 
ASEP introduction. A number of variables have been isolated. 
They are sources of information, ^es of informatipff required, 
.ppinions about ASEP and Form r science^ and views about ASE 

. and Form 1 scifence and accuracy and qongruency of these view^. 

These areas areumportant^hen^considimng/tti^^ 
education program that should be associated wi^h cuoSpilum 
development / 




I^IPACT OF ASEP ON SCIENCE TEACHERS 

The data from the second questionnaire provided information 
about teachers' views .on aspects of Form 1 science and ASEP 
after one year of ASEP availability." The non-user group can be. 
used as a comparison group . for the teachers who have made 
intensive use of the ASEP materials. 

Effect of Using ASEP Materials on Science Teachers* 
Views About ASEP 

The following four comparisoiyfbetween intensive users'^and 
non-users were made: * - ^ — - - • ^ ^ 

• Change in view of science ~ ciassroom organisation 
dimension. 

• Change in view of science — ^etlniculum structure 
— dimension. ' 

• Change in view of ASEP?^— classroom organisation 
dimension. 

• Chaurige in view ofASEP—airriculum structure dimensioa 

In the third of the preceding comparisons the impaa on 
intensive users was shown to differ significandy from the impifet 

..' ■ ■ ..... [ , . 
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on non-users. Ten out of the twenty-six intensive users were now 
adopting a less pupil-centred view of the ASEP materials whh a 
shift of at least one standard deviation/Only five out of thirty- 
three non-users had shown such a shift (X" = 4.20, Idf, p 

< 0.05).:- '■ ■ ' ' ,. , , ■ 

CHANGES INTEACHEkS' PERCEPTIONS OF ASEP AND FORM 
1 SCIENCE DURING FIRST YEAR OF ASEP AVAILABIJ^ITY 



More I 
Fupil-Centred ' 



VIEW OF ASEP 

(ClavTOom 
Organiution) i. 



Intcoiive Uaen 
March 



Non-Uien 



November 



i November 



March 




/ 



View of Icleal Science 
(Classroom Organisation) 



. More Pupil-Centred - 



the diagram above illustrates this finding. Both intensive users 
ind non-users tend to view ASEP as more,' pupil- centred' than 
Form 1 science although they differ^ignificantly in theirviews of 
ASEP and Form 1 science. (See the table on Teachef Perceptions 
of ASEP and Form 1 Science, p.24). For intensive users, using 
ASEP leads to an overall decrease in their view of it as a 'pupil 
centi-ed' approach. It is interesting to note that it is the view of the 
ASEP materials that has altered rather than any overall change m 
the intensive users' view of science. 



/^I^ffect of U^^ ASEP Materials on Science Teachers* 
Response to Semantic Differential Items ^ 

The change in responses for intensive users and non-users was 
' compared for (each of the ten semantic differential scales relating 
' to ASEP materials and each of the nine semantic differential 
scales relating to Form 1 science teaching. The change in 
response of each intensive user and non-user was set out in a 
distributiori table as shown in the next table for Form 1 science 
(meaningful — meaningless). The Kolmogorov-Smirnov two 
sample one tailed test^* was used to test the theory/that using 
ASEP materials leads to science being seen as more meaningful 
The results of a similar analysis of each of the nineteen pairs 
resulted in the following significant changes in intensive users as 
compared to non- users, 

y * 

, Intensiyf users tended to change so that ASEP was viewed as 
more superior, more unusual arid vaguer when compared with non- 
users (p < 0,05 in each case';,. ^ , . ' 

The change in view of Form 1 science (rneaningful-mean- 
ingless) set out in the table below is illustrated in the graph on 
page 28. ' ^ 

: J . 

EXAMPLE OF KOLMOGOROV-SMIRNOV ANALYSIS QF 
CHANGES DISTRIBUTION FOR MEANINGFUL-MEANING- 




LESS OF FORM 1 SCIENCE 



'. 

Foffin 1 Science 


MEANINGFUL » 


MEANINGLESS 


*Shift'of 


7I point 


1 point 0 


1 point 


> 1 point 


Intensive uiers 


2 


6 16 


1 


0 


Noii'Uien 


3 


■ 7 9 


7' . 


7 



. P<.05 



CHANGE IN VIEW OF SCIENCE (MEANIhf GFUL-MEANINOLESS) 



Mcininglcu 



Non-uiert 




Mircjtt 



KovcRiber 



SUMMARY^ 

The importance of the teacher as a source of data for curriculum f 
evaluation has been argued. A procedure was outlined for ^ 
gathering data from science teachers in the first year of ASEP; 
availability. A selection of data was presented whtch described 
science teachers' initial attributes in terms ofl subsequent use ; 
made of the ASEP materials. In the final section of the chapter die 
effect of using ASEP materials on some attributes of saence 
teachers was oudined. - > 

The chapter began by describing die teacher as the to 
die way in which a hew curriculum such as ASEp iS used in j 
classrooms. An imp<irtant justification for looking at die teacherv 
'in%e first year of diis curriculum introduction is diat a better 
under^nding of the teacher ^yill lead to more effective teacher 
education associated with ftiture, curriculum developrnent;: The , 
. resul ts set out in this chapter sugg est implications for teacher i: 
education. These have not been fiiUy deyelop/ed but we consider 
this chapter as a type of evaluation of the teacher education 
program associated with ASEP, and a source of information , to: 
rsupport effective teacher education programs in fiiture cumculumf 
developments. ' 
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EFFECTIVENESS IN 
PROMOTING PUPIL 
LEARNING 



There is a multitude of criteria and approaches which might be 
used in curriculum evaluation. One approach which has received 
particular emphasis from educational researchers is the evaluation 
of a curriculum in terms of its effectiveness in prompting pupil 
learning. The main purpose of thi^ chapter is to find out what 
pupils gained from ASEP materials during a year's science 
teaching. 



THEORETICAL CONSIDERATIONS 

Three theoretical issues need to be considered in the design of 
the study. They^are the choice between comparative and non 
comparative evaluation, problems in identifying goals as a basis 
for a fair comparison between alternative curricula, and the role 
of aptitude-treat ment interactions in curriculum evaluation 

.research. A major aim is to describe some of the problems 
encountered, and solutions proposed, in attempting to evaluate 

/aSEP in terms of piipil attainment of aims. 



Comparative or NonrConiparative Evaluation 

A curriculum evaluation issue on which agreement has.npt been V 
reached in the literature concerns the choice between cpmpara- : 
live ahd non- comparative evaluation.' dronbach* says that a . 
curriculum should be evaluated against one's ideal rather than 
against competitors. Wdch,* however, has pointed put that th^ ; 
Scisibii-maker needs toluibw, not only if aj^niculum^^ 
its^bals, but also if it achieves them more effectively than other ; 
alternatives. Furthermore, McKeachie* has reminded us that ^ . 
control group is useful in ensuring that extraneous variables ;? 
(such as the mere passage of time or taking the same test twice) do :| 
not account for the changes. ■ 

Despite its desirability, comparative curriculum evaluation 
- is not alivays possible as sometimes 'no alternative programs that 
could serve as the objects of comparison are available'.* One 
exahiple of this was the adoption of PSSC physip in Victoria , 
when a central decision was made that all classes in the State ^ 
would abruptly abandon conventional physics courses, and 
begin the PSSC course.' But die adoption of ASEP materials was: 
not laid down by a central authority so that, at the time of this 
research, some schools were using ASEP materials while others 
^sed a variety of noh-ASEP materials. 

It was therefore possible to use a comparison group in the 
present evaluation of ASEP. A comparative evaluauon would 
provide data useful to the decision-maker concerned with- 
chobsing between ASEP.and alternative materials already m use, 
and would provide control over several extraneous variables. 
This though does not leave out the possibUity of also evaluating 
ASEP against an ideal. Rather, in the present study, data abput 
changes in ASEP pupils oyer tiiile would provide a basis tor a 
non-comparative evaluation while data on the changes m ASEP 
pupils relative-to non-ASEFpugils^ basis for a 

comparative eyali ' " ^ 



- Ghoosihg Goals iPor Fair Comparative Evaluation 

.Grpbman' noted that, because new.and old curricula often have 
very different goaJs,^ standards may not be 

equally fair to the two competing curricula. This is a problem to 
be considered in evaluating ASEP, as ASEP materials are quite 

- different from conventional materials in hnany ways and, eyen in 
a noilrcomparative evaluatio n, a similar proble m would ari se 

"^becausexhWesay^ 
different ASEP pupils to cover quite different material. 

Therefore, ian important distinction needs to be made 
between content and content-free goals ' Content goals of a science 
course include the mastery of specific terms, concepts, laws and 
theories covered in that course. On the other hand, content-free 
goals go beyond the actual subject matter taught and include 
such things as intellectual skills, attitudes and interests, and 
understandings of the nature of science. This niteans that 
content-free goals v^hich are often common to science courses 
covering widely different subjea matter can provide a fair basis 
for comparing the achievement of pupils following different 
curricula ' 

Content-free goals play another important role in curriculum 
evaluation as long- term content- free outcomes which build up 
over longer periods are more important educationally than the 
short-term content goals.^Tl^refore, itwas decided thatcontent-' 
free evaluation standards woTtKJ be used in this project 



Aptitude-Treatment Interactions 

Curricular materials are- not likely to have the same degree of 
effectiveness for all pupils, due to thdr different aptitudes (such 
as age, sex, socio-economic status, intelligence, attitudes^^per- 
sonalitj')^ In fact, a;ivhole area of research, known as aptitude- 
treatment interaction research, has grown up in an attempt to 
isolate pupil aptitudes which are differentially relarted to achieve- 
ment undtr different curricular treatments. *° 



. In this study, socio- ecoriomic status (SES), IQ,and Sex were; 
chosen as aptitude variables because evidence in the hterature , 
1^ consistendy shown that they are aU relatedvto l^m^ 
outcomes. SES Was measured using GongaltorVs classificadon | 
of fathers' occupations and IQ, was measured with k modified 
version of die Ous test" , : | 

SELECTION 

" LEARNING OUTCOMES . . . ■ -v -. 

Klopfer has made the assertion: V ^ , | 

Research findings can never be more reliable than the datai g 
o^^Sthey Se based, and the findings obtmned;m a^.| 
?^hS stu&y must ahvays be interpreted mTelancpt^ g 
die quality of the instruments used to obtain the aata.^ v -| 

For diese reasons, the selection of methods to measure^^ 
outcomes in the present research was considered pfpai^miJU .^ 
importance, and certain criteria described below were usecj^n 
selecting, modifying and validating die tests. It proved impossib^ 
to select a battery of existing tests and it was necessary to modify 
existing ones and develop new tests. 

Seventeen Evaluation Scales Chosen ' i 

The next table lists the^eventeen evaluation scales choSenibrthe 7 
present study together with a source reference for each scale: The ; 
first nine scales measure various inquiry skills, the next three 
measure aspeas of understanding sdence and the remainingfiye; 

scales measure varibus attitude aims. All seventeen scales are : 

content-free. . . . l • • ' 

The TOES battery of tests consists of multiple- choice items j; 
developed specially for diis study Two trial forms were tried out- 
before use in the present research. More detailed informauon 
about the development of TOES can be found m Fraser. 



THE SEVENTEEN SCALES CHOSEN TO MEASURE 
LEARNING OUTCOMES 



^ So 



Soile 



Rsl'Sercnce 



I • .9 TEST OF ENQUIRY SKIU5 (TOES) 
TOES Part A -^ererence Materials 



Fraser 



?3 



1. Skill!: 

2. Skai2: 



Library usage 

Index wad tjibic of ooaieau 



TOES Pan B - Interpretiiit aad Proceasinf InfomuUoa ' 



3. SkUl3: 

4. SkU]4: 

.5. SkUlS: 

6.. SkU]6: 



Scales . 

Averages, percerttages and 
proportions * ' 
Cham and ubtea 
Graphs 



TOES Pirl C - Crilfcal Thinking in Science 



7. SkUI? 

8. SkU18 

9. .SkU19 



Comprehension of science reading 
Design of experimental procedures 
Conclusions and GeneralUatiofts 



iO-12 Test on UNDERSTANDING SCIENCE (TOUS) 



10. Scale P: 

11. ScaleH: 

12. Scale N: 



Philosophical scale 
Historical-social scale 
Normality orsceintists scale 



KlobferA Carrier' 
CArrier rt'al^^ 




13*15 ATTITUDE QUESTIONNAIRE 

13. Attitudes: Social implications of science 

14. Attitude Ef Enjoyment of science lessons. 

15. Attitude 1: Interest in Science 

16 ATTITUpE TO INQUIRY 

17 SCIENTIFIC ATTITUDES / 



Ormerod Schools' 
Council Pfbject^' 



26 



Meyer 



28 



Mackay& White 



29 



44 




TheTOUSscales.inTablel consk^ 
based on ah existing junior high school version of TQUS** ^ 
primary schoolyeirsion of TOIJS." The present Version has beeii j/ 
modified to make it suitable for seventh gr^^ 
original forms of TOUS give oiily a single total score, items wer^ 
divided among the three conceptually distinct sxAbscdes Hsted^^ 
in the table opposite. The reasons for choosing these three sub^ 'I 
scales and the methods pf validating thie sub^^ 
in Fraser and Fisher. ^\ Appendix II contains an ejcample 
item from each of the three sub-scales of TOys; r- I 

The . attitude questionnaire consists of items scored^ 

five- point Likert scale which fall into one of the three scal« p 
shown in die table. See Fraser arid Wrightj V ^ :^ ; ' S 

The attitude to inquiry scale is a slighdymb^^ 
the Tirtding^t about things' scale appearing in Meyer's test.?^;y^| 
Items in thistekt are scored on a scale in which pupils aillot 0:4 ^ 
votes for various activities.^* •". ■;• ; : ^-i '-;}; 

The last test listed in the table, the Scientific Attitudes test, is^j , 
a modified version of Mackay and White's TOPOSS- Self test '^|| 
While the original form of the test was developed for Australians^ | 
Grade 10 students, the present version contains numerous' | 
modificaflons to make it more readable and suitable for seventh ] 
graders. ' . ' ^ ' ■ ■ „ ' , 

The tests were chosen and modified on the basis of educational , 
importance, subjective pre-trial and statistical post-trial criteria ^.?x:\i of 
these criteria, which' have 'been described in some detail in 
Fraser,^° is discussed below. 



Educational Importance 




Cronbach'' has advQcatedstbit all tests, befoVbeing consider 
valid, should be shown to be educationally worthwhile and that no 
irnportant class of outcome be omitted. • ' V V / / 

A literature review was used to identify educationally; 
important aims, in particular those most important for science 
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' ' ' S' ' ^ ' ^ /" -''iv;> 

education." The seventeen evaluation scales were then chosen to , |l /\ 

' provide a reasonable coverage of thesp aims. A high congruence 

: ' was shown fo exis^t between ASEFs stated aims and those stated ^ ^ * 

; a3 important in the litierature,^* and so the present study can be , > * 

' used to evaluate ASEP based on its stated goals and on an ideal set ; 7, 

, of goals taken from' the literature. . . ' t * ^'-; 



Subjective Pre-Trial Criterjia 1 

A panel of science education and , education measurement 
experts soputinised the tests before they were used. Each tesVwas 
assessed/according to a number of subjective criteria such as face 
validity, potential readabilicy, suitability for Australian children, 
ease of administration and freedom from dues, ambiguities-and 
other item faults" arid a number of items wer6 reworded or 
■ omitted.^ •, ■ ■ J ■ ■ '-yy:'. ■ 



Statistical Post-Trial Criteria 

Before being used, all the evaluation tests were given a trial run 
with seventh graders. The sample sizes varied from one hundred 
for the scientific attitudes scale to four hundred for the TOES 

• tests. Information from these trialis was analysed to give informa- 
tion abput three important statistical attributes of each test: 
internai c(msisteruy; discri These statistics 

gave indications of the overall effectiveness of scales and enabled 
faulty items, to be identified and removed. : ' 

■ Cronbach" says that a test score must have substantial 

. jntemal coiisistency so that each item measures the same thing 
measured by the rest of the test For the present senes^of scales, 
the Cronbach a reliability coefficient has* been chosen as the 
index Q|4nternal consistency. The table below shbWs the niinib^ 

r of items contained in each scale together with the yaluie of the 
coeffi cien t ob tai ned from ; th e trial adminis trati on of each scale. 

^ This table indicates that the a coefficients ranged from 0.55 to 
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0.85 and had a median value of 0.69. The values of the a coeffic- g 
irat were^ 
testsy especially 2^ 

•'items. ' -i'^ • ' 



. . . 

SCALE STATISTICS OF/I>nrEIWAL.CpNSISTCNC:Y|g.|^^ 



\r' 'J: ■■p-"; ■ • 


i;.;,-.; Number of items ^V;?;;'^ 


V;.^x^(Crontich&t^. 


TEST OF ENQUIRE SKuiiS^^^" , 






toes'-m',.'\/V.'-'-:;:^-: :-*-■ . r>-" ' ' 


■ ..:I0-^: 




'toes' ;'.:2'-;;c S:"--- ' ] ■^v,!::'- 






-TpEs:-':3;\*' . 






TOES-' ■4. ■ • 






■TbES'..'--5'-''' 


■■ u» ■ ^' "'. - 




TOES 6 


10, 


■:;■^-^.V' o.77f"''i: • 


TOES 7 ^ 


10 


: V.Vb.-65' ■ 


TOES 8 , * 


V 10 


0.60 ■ ." 


TOES 9 : 


. . ' ' 9 


■'o-70 


TEST ON UNDERSTANDING SCIENCE 






TOUS P ' 


12 . 


■ • . ■ ■ ■ ,^■ -. 

■ ■' --'viO^S 


TOUS H , 






"tous N , V 


• ^ -6 . ■■'■,.7 ' 


''■■■•.0.60 


ATTITUDE questionnaire 






Altilude 5 .; V 


;.'8;' ■ 


0.81 


Altllude E ..: . . 


- y-j ■ 


.-• ■■'^:0.85 V: 


Altitude 1 :y 




' 0.80 ! 


ATTITUDE TO INQUIRY . ' 




0.67 


scientific ATTITUDES , 


n . 


■■■ 0.63.- /■ 

■1 



mm 



/ 

/ 



' As well as being internally consistent, each scale in 2itest series|;| 
should possess discriminant validity.** This critierio^ deipands|i 
that each scale .measures a unique construa not^eaiu 
other scales in the tests. The statistic chosen as an index^^^o^^^ 
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fejifdiscrinlinant validity for this series ivas^ die interrcorirelationl/^ v^^*^-;' ^ 
y : between scales." An examination of inter-correlations aniong t ^^ V ^ 

scales, which is shown in Fraser^* indicated that all corrections 
■^;yjfere sufficiently^ low to satisfy the criterion of discriminarit/ 
?y^^- validity and niaintain each of the seventeen scales as a separatie 



measure. 



The third statisucal post- trial criterion -r- sensitivity^^—^^^ ^ 
2ii::^index--pf-a->tests--useftilness-in~detecting-pupil^changes.-^^ 
■ example, ihsensitivity of an evaluation test could give, rise to a , 

I ceiling effecf^ which occurs when subjects obtain near the mspdmum ^ - 
s| : : score on a prertest and it is thyerefore virtually impossible for such 
jE students to improve on the post tests/even if they;have in faa 
S improved their level of attainment on the construct underlying : ■ 
the test Thte scale statistic ch as an index of sensitivity for the ( 7 
5 present tests was the distribution of pupil total- scores cih each 
l__ scale*° which are provided in Fraser.*^They indicate that; on the 
whole, scores obtained on each scale covered ai lsurge proportion 
: of the available range and that the teists were sensitive enough. 



DESIGN OF THE STUDY , 

■ - ■ ■ ^ ■ , ........ 

In line \yith the decision to conduct a comfJarative evsduation 
. both an ASEP and a control group wereemployed in the present 

:Study. The ASEP treatment was defined broadly with the only 

■restriction being that ASEPmaiterials be used exclusively during 
I the time of the study and that the sequencing and timingof ASEP 
f units. were left to the discretion of each teacher. The control 

• group could use any sd as long as 

they iyei^e not ASEP materials. « \ 

The table on page 42 shows that the total s^ple consisted of 

1,158 seven'fh grade pupils in forty-six different glasses, each 

in a different co-edu(Gational ,^ 

metropolitan area. The schools \yere spread widely through the , 
different geographic and socio-economic areas of Melbourne 



and, of the fbrty dasses; Wenty^c^ 

twenty-s&ere in the control group; The actual unit of statistical 
: zmalysis us(fid in the study vyas the dasf aptitude re^^ This 
sampling invblved the division of pupils wthih ea^^ 
eight grojipsia^tco IQ^and sexi (That is, the 

eight groups vvo^blcl be: hi|h SES high IQ; — male, higH SES — 
■highIQ^T-^female)y>v:i^^lpW^^ "~ " 
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iSAMPLE SIZE 



^ r p Group 


' ClasKt ' 




Class Xptitudcv: 

Cells'-:: 




ASEP 


■; 20 " 








CONTROL 




646 




'f 


TOTAL 


■46': ■ 


1,158 

















The study lastedforawhole school year for two reasons. First, a 
major criticism of past research in-science education has been the 
short tinie of investigation." Second; it was thought that pupil 
changes on content-free learning outcornes would be relatively 
slow so that a comparatively long time between pre-testing and 
post-testing would J)e desirable. The seventeen evaluation scales 
used in die present research were given as pre- tests dunng 
March, 1974 and as post- tests in October. 

RESULTS 

The da\a were analysed in two ways to throw light on two 
different^qiiestions. First, data abom^^^ changes in each 
learning outcome from pre-test to post- test jproyided a basis for a . 
non-cornparative evaluation of ASEP. Second, a comparison of 
the changes experienced by ASEiP pupils with those experienced 
by non-ASEP'pup*ils provided a basis for a comparative evaluation. 

^ 9 
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SIGNIFICANCE TESTS FOR CHANGES DURING TI 
FOR THE WHOLE SAMPLE 



[E YEAR 



Safe 



1. TOES I 

2. TOES 2 
TOES 3 
T0ES4- 
TOES 5 
TOES 6 
TOES 7 

8. TOES 8 

9. T0ES.9 



10 TOUSP 

11. TOUSH . 

12. TOUSN 

13; AlliludcS 

14. AttriudeE 

15. Attittidel . 

16. Attitude tu 
Inquiry /. 

17. Scientific 
Attitudes 



Haxirouin 
Score 



10 
9 
10 

"8" 

10 
10 

^10 
9 



6 
6 

.40 

35 
30 



32 



Mctn 



Post 



5.6 
6.0 
- 5.1 
-2:5- 
6.5, 
4.6 : 
5.9 

ii 

4.3 

5.3 
6.7. 
3.5 

27.3 , 

23.2 

18.0 



22.3 



6.0 



. 6.0 
6.0 
/ .5.9 

;^ ^3.0- 

6.8 
5.2 
6.2 
- 5.5 
4.6 

5.8 
7.1 

26.8 

2i.9 
16.9 



22.6 



6.2 



4.2*** 

0.5 

8.2*** 

-6.7*««-- 
2.8«- . 
5.5««« 
4,2««« 
2.4« 

5.4«« 
3.0«* ' 
7.8«« 

^-2.3* 

"5.5«** 
-6.6«*« 



1.2' 



2.6** 




•p<.05 



••p<.oi 



•p<.00l 



This table shows rhe pre-test mean and posMest mean of the 
whole sample- for each learning outcome together with the 
results oft tests for dependent saniples for differences between 
pre-test and post-test performance. This .data indicates that 
differences between pre-test and post-tes\ scores were signific^t 
for fifteen of the^seventeen scales with the two exceptions being 
TOES 2^hd the Attitude to I nquiry-scale- AlLtto scales of the 
attitude questionnaire (social implications of science, enjoyment- 
of sciencd lesson? and interest in science) showed that the 
significant changes over the year were k fact df^c^^^^ positive 
attitudes. Similar disturbing findings of deterioration in attitude 
duling the study of other science oinimlar materi^^ been 
reported by Mackay/' Welch and Walberg** and Choppin.*? 
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Relative changes in ASEP and Non-ASEP Students 

The next table shows the relative changes experience^ by ASEP 
and control pupils. It provides values from multiple regression 
^alyses involving a direct comparison between the post- test 
performance of ASEP and' control pupils (while statistically 
controlling for pre- test and other variables). It also gives compari- 
sons of the performance of pupils of higher and lower SES, of 
higher and lower . 

SIGNIFICANT F VALUE FOR EACH LEARNING OUTCOME 
POST-TEST USING INSTRUCTION (ASEP/NON-ASEP). SESiflQ, 
AND 5eX AS PREDICTORS, AND CONTROLUNG FOR PRE- 
" TEST AND ELEVEN INTERACTIONS ' 



Sode • . ' ' 


ASEP/Control 


F vihics . 

SES IQ 

<», • •■ ■ 


Sex 


1. TOES I 

2. TOES 2 

3. TOES 3 
4 TOES 4 

5. TOES 5 

6. TOES 6 

7. TOES 7 

8. TOES 8 A 

9. TOES 9 




3.9* 


50.0*** 
30.3*** 
' 50.5V* 
^9.4«** ':■ 
57.5*** 
64;2*** 
56.5*** 
73.0*** 
50.1 


G 43* 
B 5.6* 


10. TOUS P 

11. TOUS H 

12. TOUS N ' 




5.0* 

\ ■ ■ 


39.9*** 
98.7*** 

32.7*" 


G 6.9** 


13. Allllude S 

14. Aliiiude E 

15. Allllude 1 


A 13.7V' 








16. Aliiiude lo Inquiry 










17. ScientiflcAlliludes 






20.3*** 


B 7.8**: 


•ip<)05. p<.Ol, 


• p<.OOI 









A ASEP jupcrior. G Girls superior. B Boys superior 



In all cases for which F values appear for SES or IQj j^upils of 
: higher SES and higher IQ^out-performed pupils of lower SES 
5and lower lQ, respeaively. ^^^^^^^ j ;J ' ; - 

THE SES-TREATMENT IhnrER^ 



Raw 

Scores 
For 

Attitude E 

23 



22 



21 



^ighSESASEP ^ ^ ^ — 




^J*re-le4t 



J- 



Testing Occasion 



Post - test 



4 

The change in performance of the ASEP group over the year was 
Significandy different from that of the control group for only one 
of the seventeen leammg outcomes, namely attitude £ (enjoyment 
of science lessons). The interpretation of this finding is illustrated 
by the simplified plot of raw scores shown in the diagram above 
This shows that, while the ASEP and control group experienced 
similar enjoyment of science lessons at the start of the year, both 
groups experienced some decline in enjoyment during the year. 
However, the ASEP groiip underwent a much smaller decline in 
enjoyment during the year than the control igroup; There exists a 
SES- treatment interaction for Attitude £ " 
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The table on page 42 also shows that there were significant , 
relationships betweefi certain learning outcomes and SES, IQ, 
and sex. Pupils of higher SES experienced a greater improvement 
than pupils of lower SES on TOES 8 (design of experimental--^ 
procedures), TOUS P (Philosophical scale) and thejidefitific 
Attitudes scale; pupils of higher IQ^ experienced a greater 
improvement than pupils of lower IQ^on all twelve cognitive tests^ 
and on the Scientific Attitudes scale, boys experienced a greater 
improvement than giris on TOES 4 (Averages, percentages and. 
proportions)^ and the Scientific Attitudes scale;- girls experienced r 
a greater improvement than boys on TOES 2XIndex and table of 
contents) and TOUS N (Nornisdity of scientists). As well as the 
significant difference between ASEP and control pupils for. 
Attitude £ (enjoyment of science lessons), a significant aptitude- 
' treatment interaction also occurred for the Attitude £ scale. The 
interpretation of this interaaion is also shown in the diagram on 
page 457 Changes in enjoyment of science lessons during the year 
in the control group were quite similar fpr pupils of high and low 
SES. On the other hand, changes in enjoynient of science lessons 
in the ASEP group varied markedly with pupil SES. The broken 
lines in this diagram indicate that, while ASEP pupils of high SES 
experienced a slight increase in enjoyment of science lessons 
during die year, ASEP pupils of lower SES experienced a marked j 
decline in enjoyment 

PROBLEMS IN INTERPRETING FINDINGS 

The classes in the sample .were neitiier randpmly selected nor 
randomly allocated to experimental and'control groups. Never- 
tiieless, it was found that\he ASEP group was not significandy 
different from tjhecontfbl^n SES, IQ, sex or any of the seventeen 
pre-test measures On the other hand, because some schools and 
teachers between the teacliers in' the ASEP and the control 
group cannot be completely ^dismissed. * ^ 

Although the sample was notrandomlychosen, it was large, 
covered wide geographic and sodo-economjc areas and appeared 
representative of co-educational high schools in the Melbourne 



metropolitan area. Therefore, generalisations to other - such : 
^qhools^c^^ with reasonable confidence although ^ 

generalisations to other ;types of schools or to schools outside 
Melbourne would be more^dangerous. : > ( 
V ; Despite thV imp 

appredaa:ed that many important content goals also exist and it 
must be remembered that content goals do not provide a fair 
.basis for comparisons between competing curricula ^^e^ ' 
ofASEP in terms of achievement of content aims would requirea 
different approach/^ and no inferences about achievement of 
content goals can be made. ^^.^^ ; . \ i . 

SUMMARY AND CONCLUSIONS 

This study is the first major inquiry involving an . evaluation of 
ASEP based primarily on pupil learning outconies. As such, a 
certain amount of ne^ ground has necessarily been covered in 
resolving important conceptual and methodological problems. 
Therefore, the significance of this studyishould be seen as mUch 
in terms of its methodology as in its actual finjiings. 

An important aspea of the study was the development (or 
modification) and validation of a series of seventeen evaluation 
scales suitable for use with Australian seventh ' graders. In 
particular, the fact thareach of these scales is content-free enables 
fair comparisons to be made between pupils following, quite 
different curriculum materials. 

When the battery of seventeen scales was administered as 
e-tests and post-tests to a sample of 1,158 pupils in co- 
educational high schools in the Melbourne metropolitain area,' it 
was found that the total sample undenvent significant changes 
during the year for fifteen out of sevent%n aims considered^The 
significant changes which occurred for three atdtudinal measures, 
howeve;r, were in fact a deterioriation in' positive attitude during 
the year. When the performance of AiSEP and control classes was 
compared using multiple regression analyses, it Was found that 
the two groups differed significaridy on only one outcome, 
namely Attitude £ (-enjoyment of science lessons). While both the 




ASEPand the\:ontr6l group experienced a decline in enjoyment 
of science lessons during the year, the decline in the ASE^ group 
was considerably smaller than in the control group. Furtheimore, a 
significant SE5-treatment interaction emerged for the same 
sade: whereas changes in enjoyment were almost independent 
of SES iii^ the control group, ASEP pupils of higher SES 
experienced a small increase in enjoyment while ASEP pupils of 
lower SES experienced a decline in enjoyment A significant 
relationship was also found between SES and changes in three 
learning outcomes, between IQ^and changes in thirteen learning 
outcomes and between sex and changes in four leaming out- 
comes. , 

It is of interest to look more closely at the educational 
significance of the one statistically significant difference in 
leaming outcomes found between ASEP and non-^ASEP pupils. 
It was found that the ASEP group experiencedan arrest of 1.7 
raw score points (or about one-third <)f a standard deviation) in 
the decline in enjoyment of science lessons experienced by the 
control group. It was also found^ that the treatment variable 
accounted for 2.7 per cent of tht variance in Attitude E post-test 
scores, after the variance due to pre- test, nSES, IQ^and sex had 
been removed VV^hen these data are consider^'d in conjunaion 
,vith the fact that second generation science^^rojects set out to 
foster piipil enjoyment of science, the present finding of a 
.significant difference between treatment groups for Attitude E 
cvssumes educational significance. ) 
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THE LEARNING 
ENVIRONMENT AS 
A FOCUS FOR 
ASEP EVALUATION 



This chapter begins by outlining the way in which ASEP 
appeared to describe the teacher's role when using the materials, 
then describes an appropriate evaluation procedure and its 
results in twenty-three classrooms using one ASEP unit 

THE ASEP VIEW QF THE TEACHER'S ROLE 

ASEP developers produced materials intended to stimulate 
teachers to re-examine their roles as science teachers and the 
teacherwas required to become rhe major decision- maker. With 
the project producing modules (units) of work, and the organisa- 
tion of units with a certain amount of optioi^ material the 
teacher was being called on to make curriculum decisions at all 
levels. Whether good or poor choices were made, the task and 
responsibility were with the teacher.' The teacher was given the 
opportunity to move away from a role as an * information 

4 



controller' to a role as a facilitator of student learning. The 
materials were designed to allow students to organise their own 
activities to a greater extent than other science curricula. The 
gTi/ide for teachers^ clearly sets out many of the characteristics of 
the classroom in which the ASEP developers felt the materials 
/ would be used most effectively. The environment and organisation 
/ of the science classroom was clearly an important focus during 
the ASEP development 

CURRICULUM EVALUATION AND THE 
LEARNING ENVIRONMENT OF THE CLASSROOM 

^ Just as the nature of the science classroom has been a focus for 
the curriculum developer so \t can also become a legitimate 
concern for the curriculum evaluator. Walberg* provides some 
justification for studying classroom variables when he proposes 
. his mddel for learning in which three groups of variables — 
aptitude, instructional and environmental variables — are sug- 
gested as making major contributions to learning. 

Walberg* argues that environmental variables canJ:>eTnan- 
ipulated by the teacher in contrast to aptitude variables. Such 
manipulation^could be expeaed to alter learning outcomes.*^ A 
number of studies have been designed to examine the relationsbip 
between environmental variables and learning outcomes and 
various dimensions of the learning environment have been 
measured by using pupil self-report inventories such as the j 
Learning Environment Inventory (LEI). 
' Increasing'use is being made of student self-report inven- 
tories in studies of classroom learning environments.® Anderson 
and Walberg suggest a number of reasons for this: 

• Inventories provide a very economical way of gathering 
classroom information compared with alternative methods 
such as classroom observation. 

• Inventories provide a more valid way of gaining inform- 
ation about classrooms than methods involving outside 



observers. Pupils form a group of respondents likely to 
be very sensitive to the significant and .unique features of 
a classroom. An outside observer, although trained and 
systematic, is probably less sensitive than the pupils 
involved. 

In classroom studies the learning environment has been 
used as both an independent and dependent variable. A number 
of studies have shown the learning environment (as measured by 
instruments similar to the LEI) to be a significant predictor of 
important learning outcomes,' Findings like this are critical if 
there is going to be continuing emphasis on 'shaping* the class- 
room environment in curriculum development It is essential 
that the roles of teachers and pupils being suggested by curricula 
such as ASEP be shown to be likely to lead to desired pupil 
outcomes, , 

It hai^ also been demonstrated that certain characteristics of 
the class can be used to predict the classroom environment 
Studies using learning environment variables as dependent 
variables are essential. 

In Walberg's model of learning* increasing attention to 
environmental variables is justified by arguing that this group of 
variables is subject to manipulation and therefore important in 
learning. If this is the casewe would expect tobeable to show that_ 
certaii? characteristics of^the class are related 'to the learning 
environment . 

Teacher educators and curriculum developers have to be 
concerned with the factors whiqh seem to be associated with 
pau'ticular learning environments. 

The relationships between learning environment variables 
and various predictors on the one hand, and learning outcomes 
on the othtr can be illustrated by using the Getzels and T^helen 
modeP which forms the theoretical basis for LEI development 

The diagram on page 56 illustrates Getzels and ^helen's 
concept of climate as being the result of a complex relationship 
between the roles of the participants in the classroom and the 
individual personalides of the participants. 
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RATIONALE FOR LEARNING ENVIRONMENT 
EMPHASIS IN CURRICULUM DEVELOPMENT 
AND EVALUATION 



® 

Tetcher 



©. 

CETZELS AND THELEN MODEL 



••Curriculum \ 




Institution — ^i^Rolc ■ » Expectation^^ - , | 

groupi ■ * Climate) Intentional Observed ! 

t J ^ha*****^',' 



■■ V ■■■■■■■ "X 

Pupils and teachers occupy certain social positions related to 
the way science is taught in a school. As the roles arid personalities 
of the participants vary so will the classroom climate; Looked at 
in this way programs such as ASEP Which attemptto change the 
classroom are in fact doing 50 by asking teachers to I'e-exariiine 
their roles as science teachers. The resource materials allow 
teachers to move from being major organisers of learning in.the 
classroom to being facilitators of learning by hayi^^^^^ 
work in groups arid allowing the written resources to take over 
some of the organising funaion for the studentg^' 

The remainder of the chapter is concerned with one aspect 
of the role of teachers and pupils using ASEP materials and the 
effect on classroom climate, (Relationship fl in the previous 
diagram). Before describing this study however, the previous 
diagram can be used to place this aspect of the study into an 
overall context The relationships represented by y4 in the 
diagram show the classroom climate (learning envirohment) 
variables as independent variables in the prediaion of learning 
outcomes. In Chapter* 5 of the report, this relationship is 
described in mbre detsuL i) represents the impact of A5EP on 
teachers, the subject of Chapter 2. C represents the effea ASEP 
introduction has had on telicher and student roles in the 
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classroom. The important teacher education questions of how 
teachers might be encouraged to consider and adopt new roles 
would be included in C of the previous diagram. ^ 

I 

EFFECT OF DIFFERENTTEACHER ROLES ON THE 
CLASSROOM LEARNING ENVIRONMENT 

Design of the Study 

^ Of the 151 respondents to the first questionnaire (see Chapter,2) 
twenty- three Grade 7 teachers were asked to use the ASEP unit 
•Places for People' atthestart ofTerm III, 1974. Each teacher was 
given: * 

• a copy of 'A Guide to ASEF; / 

• a copy of the teachers' guide for the ASEP unit 'Places 
for People'; 

• a class set of the student books for Tlaces for People'; 

, • a class set of the ASEP service unit 'Heat and Tempera- 
ture'; 

• copies of student charts, questionnaires; etc 

At the meeting with each teacher the design of the study was 
explained (see diagram below). However, no instruaions were 
given as to how the ASEP unit should be presented; thiis was left 
to the teacher. 

Methods Used in the Study 

The SCI is a modification of the LEL The modifications were 
designed to make the inventory suitable for Grade 7 students and 
the final form is the result of preliminary trials. The nine scales 
included were selected because they seemed to represent im- 
portant features of ASEP classrooms as described by the ASEP 
developers.'" The table on page 56 sets out the nine scales with an 
example of an item in each scale, the Cronbach a reliability 
value for dach scale and the number of items which make up each . 
scale of the SCI. 

■ • ■ 64^^ 
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DETAILS OF S.C.I. SCALES \ 

I \ 



Scak 
(Name of limilar 
LEt Swk) 

FormMlity 
Indivlduiility , 



Speededness 
(Speed) 

Envirnnmentil 

SuitiibUity 

(Environment) 

Go«l Directedneu 
(Goal Direction) 

Satisractlon 



Dlsorganiution 

Difficulty 
Competitiveness 



Sample Item 

Students rollow* strict rules 
in science classes. (♦) 

Students ctn chooie to itudy 
dirfereni science topics 
that interest them. (>) 
The work in science is 
covered too quickly. (>) 
Science lessons are held in 
I room which allows us to 
carry out our science 
activities easily. (>) 

Students have little idea 
of what the real point of 
studying science is.(-) 

' ' After the class, the students 
reel science lessons are 
worthwhile. (+) 

* There are long periods 
during which some class 
members do not know what 
10 do. (+) 

The questions asked in science 
are often dirficult.(+) 

Students often race to see 
who can finish first. (*) 



Cronbach 
flC Reliability 

0.53 
0.5SI 

0,72 
0.70 



0.66 



0.83 



0.74 



0.48 
0.60 



fit' 




DESIGN OF THE STUDY 



*Hcat and T«mp«ralurf ^riMnttf—fk' 
Service Unit 



-4 



Pr»-to«i(5CI) ^ Student ciiarti of activities Poet-teat (SCI) 



: * Tcacheri were informed that only p. 1.19ortheserviceun)t containedlnrormationdirectly 
relevant to Jh« unit PlaceM for People. 

Students were asked to respond by circling one of four 
alternatives: strongly, disagree (SO), disagree (D), agree (A), 
strongly agree (SA) and these were scored 2/4, 6 and 8 
respectively for a positive item shown (+) in the table, and 8, 6, 4 
and 2 for a negative item, shown ( — ) in the table. No response or 
rnultiple responses to an item were scored 5. It was therefore 
possible to obtain a score for each pupil in each class for each of 
the scales in the SCI. 

The Student Charts 

Students were asked to keep a record of their activities in each, 
lesson to show how the ASEP unit was presented in each dass. 
Pupils completed their own charts at the end of each lesson by 
placing a tick against the part or parts of the unit they had been 
working on in that lesson. When the charts were returned at the 
end df the unit it was possible to collate the informadon from 
each pupil in the class to form a 'pattern' of the way the unit had 
been presented in each.class. Two examples of these * patterns' 
are included in Appendix IIL For each lesson it was possible to ^ 
determine tl>e number of different parts 9f the unit that had 
received attention from pupils." The mean number of activities 
per lesson was then calculated. 

The classrooms were grouped according to the mean 
number of activities per lesson. The next table summarises the 
way in which the. classrooms were grouped and it appears that 
science teachers wore implementing the ASEP unit in three 
distinct ways: 
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• Wide student choice — in these classrooms students 
' were involved in a wide range of activides from the first 

lesson onwards.'" 

Less student choice — in these classrooms students 
were involved in very few activides in the inidal lessons in 
the unit Towards the end of the unit a wider variety of 
acdvities was evident 

• Limited student choice — students were involved in 
very few activities in each lesson throughout the^unit" 

CLASSROOMS GROUPED ACCORDING TO MEAN I^UMBER 
QF ACTIVITIES PER LESSON WHEN USING THE ASEP UNIT 
\ ^ TLACE? FOR PEOPLE' 





Mode of Implementation 


Mean Na of Activities Lctson 


Ran|( 


1 . Wide student choice 
(S classes) 


8.7 


7.8 to 9.7 


2. Less student choice 
(5 classes) 


7.0 


6.8 to 7.3 


3. Limited student choice 
(7 classes) 


4.6 


2.2 to 5.8 



Note: Six classes of the original twenty three classes could not be 
included in this analysis. ' 

• Two classes did not complete the ASEP unit because of 
teacher changes. . 

• Two classes used the ASEP unit in a General Studie? 
program and the activities were linked with other 'non- 

, unii^ activities. — - 

• Tw6 classes h^d small student numbers (< 15) which 
meant that the mean number .of activities per lesson 
could not be compared with other classes (> 23). 
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Method of Analysis 

In a nujS^iber of studies using the LEI scales as dependent 
variables a number of predictors of classroom climate have been 
reported.^* Although this study intended to examine the^effect of 
the method of presentation on the ASEP learning environment, 
it was necessahy to also consider other variables that could affect 
the learning environment Previous studies show that the learning 
environment is likely to be affected by curriculum, class size, 
grade level, girl ratio, socio-economic status and ability leveL^* 
The study design controlled the first four, groups of variables to 
some extent (a particular ASEP unit used in a Grade 7 co- 
educational class in Victorian high schools). SES has been sho\vn 
to be associated with students' response to *compeddon' items. 
No attempt was made to control this variable in this study. Ability 
level has been measured in several ways in past studies (cognitive 
pre-test, IQ, grade point average, etc). Ability level seemed to 
predict difficulty, disorganisation, speed, formality and goal 
direction. No design control was attempted but it seemed to be 
important to check whether ability level was a significant factor in 
students' perception of the learning environment 

This was attempted by asking teachers to indicate the 
students who appeared to have most ability in science, and the 
students who^had the most difficulty in science at the end of the 
unit 

Three factors were considered in analysis of the findings: 
mode of implementation; ability level; and change in perception 
of learning environment during the ASEP unit (time effect). 

Results and Interpretations 

The results of the analysis are set out in the next tableland some 
of the significant results are represented in the following six 
figures. A summary of the findings is presented below: 

• The presentation of the ASEP unit seemed to be associated 
with significant changes in five of the nine scales making 



up the SCL Over the period of time the ASEP unit 
'Places for People' was presented students reported a 
significant inaease in individuality 3X\d goal directedness. 
There was a significant (fecr^as^ in satisfaction, difficulty and 
competitiveness ( see Time — Faaor C in the"table and Figs, 
A, Dand E). - 

• The wayASEPwas irnplemented seemejd to beassociated 
with significant differences in student perceptions on five 

of the nine scales which make up the SCL Overall wide 
choice classrooms are seen ''as more individualised, less . 
speeded, more satisfying ^nA less disorganhed^^t^^ 
choice classrooms (see Figs. A, B/ E and F). The less 
choice classrooms were perceived as mor^ goal directed by 
these students than students having the unit implemented . 
in other ways. (See Mode of Implementation — Factor A 
in the table and Fig. D.y \ • ' 

• Ability level proved to be a significant vatiable for one of 
the scales of the SCL Low ability 5(ud^w^i reported the 
Unit presentation as being mor^ speeded than other ability 
groups. ( See Ability Level — Factor B in the table and Fig. 

B). ' ^ 

In one scale the F value of the A x C interaaion (Mode of 
Implementation ancl Time) reached significance at the five per 
cent level. Fig. C illustrates this interaction — the classes in which 
a large number of activities occurred showed a slight increasejn 
" eiivif onmental suitability while the classes with very few activities 
per lesson showed a decrease in environmental suitability. 
The physical conditions appeared less suitable in classrooms 
where ASEP was, used with a more restricted student choice. 

CONCLUDING COMMENTS 

An argument has been presented for evaluating ASEP by 
gathering data about the learning environment' where ASEP 
fe?irerials are being used. A study was described which attempted 
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Analysis of Variance Table Summarising the Effects of Mode of Implementation and Ability Level on ; 
Student Perceptions of Nine Aspects of Learning Environment During Presentation of ASEP Unit: 
'Places for People'. . i 



ANOVA SUMMARY 



DEPENDENTVARUBLE 



SOURCE 


df 


FORMALnV 


INDIVID: 


SPEED 


ENVIR. 
SUITABIUTY 


GOALDIR 


■ 

SATISFACnoN 


DISORG. 


DIFFICULTY 


COMPETITION 


Mode of 

litipteinenution A 


2.42, 


2,33 NS 


0 




1.72 NS 


3.42* 1 


L4,68»| 


3.68* 




■ .. ■ 


Ability Level B 


2.42 


I.90NS 




4F 




2.30 NS 




V , 
1 ■ 


2.01 NS 




TimeC - 


li42 


(+)2.6I NS 


't)l7.41"» 




H2,49NS 


(+)5.02* 






(-)i.83 


1 


(-)16.62*" 


AxB 


4.42 




I.27.NS ,;' 
















AxC 


2.42 








HE 




1,50 Ns: 


2.13 NS ,; 








BxC 


2,42 






1.74 NS 








1 






AxBxC 


4.42 






l.SJI)IS 












t 


1 . 

4 

2,42 
1.42 
4,42 


CritlcilFV 
Probibility 


ilueifor 

Uvclt 






i ■ . ; 


(•) 

0,10 


0,05 


0.01 


\ 








W 


ReprejenlsiflCteuelnKi 


Je value 

ll»'Vl1ll(' 


2,43 
2,83 
2.08' 


3.22 
4,07 
2,S9 


S.1S 
7.27 
3.80 


\ * 

; ^ ^ 

1 

1 


' \ 
1 / 



GRAPHS TO ILLUSTRATE THE SIGNIFICANT EFFECTS' 
FROM THE PRECEDING TABLE 





INDIVIDUAUTY 


FifurcB 


SPEEDEDNESS 


33.00 
31.00 
29!oO 




24.00 
22.00 


✓ 


"IT f¥l 
L /.UU 








• 25.CC 




20.00 






Pre ASEP Post ASEP 
Unit Unit 




' High Middle Low 




Time 




AbiUtyUvel 




ENVIR.SUtTABILrTY 


Figure D 


GOAL DIRECTEDNESS 


\ 35,00 

\ 

33.00 
31.00 
.29.00 
27.00 
.25.00 




33.00 
31.00 
29.00 
27.00 
25.00 


1 _ J ■ 




Pre ASEP Post ASEP 
Unit Unit 




. Pre ASEP Post ASEP 
Unit Unit 




Time 




Time 


Figure E 


SATISFACTION 


FIpircF 


DISORGANISATION 


3:. 00 




31.00 




20.00 




29.00 




27.00 




27.00 




25.00 


i 1' 


25.00 






Pre ASEP Post ASEP 
Unit Unit 




Pre ASEP Post ASEP 
Unit Unit 




Time 




Time 



Wide student choice • Less student choice ---- Limited student choice 



to determine the effect of presenting an ASEP unit on student 
perceptions of the learning environment Incorporated in this 
study was an attempt to measure different modes of implemendng 
the ASEP unit related to the different roles of teachers and pupils 
in selecting the content of the unit It was possible to determine 
different student perceptions of the learning environment related 
to presentation of the unit (Factor C), the way in which the unit 
had been implemented (Factory4) and ability level of the students 
(Factor 5). i 

With the introducdon of science curricula which' suggest 
appropriate roles 'for teacher and pupils it seems necessary that 
the effect of di^k^Ftnt methods of i^nplementadon on the 
resulting learnin>c( environment continue to be investigated. The 
teacher education associated with ASEP has sought to make 
teachers aware of appropriate roles for teacher and pupil io 
occupy when using ASEP materials. Empirical evidence related 
to the effectiveness of various methods of implem'entation may 
be useful in further clarifying the arguments for using curriculum 
materials in particular ways. 
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CHAPTER 5 
FURTHER RESEARCH 

The research outlined in this chapter consists of two aspects. 
Firstly, a comparison will be made between the classroom 
climate perceptions of ASEP and non-ASEP pupils. Secondly, 
the relationship between learning outcomes and classroom 
climate perceptions will be explored., , _ *^ 

COMPARISON OF THE CLASSROOM CLIMATE 
PERCEPTIONS OF ASEP AND NON-ASEP PUPILS 

This research can best be understood by comparing it with the 
study involving learning outcomes, described in Chapter 3. 
While the previous study involved the relative changes in 
leaning outcomes experienced by a sample of ASEP and non- 
ASEP classes during a year, this research involved a comparison 
of the classroom climate perceptions of ASEP and non^ASEP 
pupils at the end of second term. Whereas the study in Chapter 3 
•involved a sample of forty-six classes, this research involved ten 
ASEP and ten control classes drawn from the original forty-six 
class sample: this sample is described in more det^l in the 
following table. In both the present and the previous study, each 
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class was drawn from a different co-educatio|ial. high schobl in- 
the Melbourne metropolitan area, and both the ASEP and the 
control group consisted of schools which were representative: of 
the geographic and socio-economic areas of Melbourne. As in 
the investigation described in Chapter 3, the present study 
included the three aptitudinal variables of SES, IQ^and sex, 

SAMPLE SIZES FOR ANALYSES EMPLOYING 
CLASSROOM CLIMATE SCALES 







' SAMPLE SIZE 








PuDik 


Cl«»aptitu<fe 


Croup 


CUSKS 


■ ceOi — 


ASEP 


. 10 


266 


'^^ 


Control 


10 


265 


77 


Total 


20 


S3I 


153 



Classroom climate was measured in this research using a 
slightly different version of the science class inventory from that 
described in the previous chapter. These climate scales were 
validated with the sample of 531 pupils described in the previous 
table using the statistical criteria of internal consistency, discrim-' 
inant validity and sensitivity. The next table lists the nine climate 
scales involved in the present research together with the number 
of items in eacj/scale,oScale reliability values and the scale inter- 
correlation matrix. 

The teJchniques to analyse the climate data were analogous 
to those/Used for learning outcome data. It will be recalled that 
multiple regression analyses were employed in comparing the/ 
post-t^st performance of ASEP and non-ASEP pupils, while 
controlling statistically for the corresponding pre- test, for SES, 
-IQ^and sex, and for various interaction terms. Similarly,^in the 
present analyses, scores on each classroom climate scale in 
August were compared for ASEP and non-ASEP pupils, while 
» controlling statistically for SES, IQ, sex and interactions. The 
class aptitude cell sampling unit was again employed as the unit 
of statistical analysis but no pre-test climate data were available. 



NUMBER OF ITEMS. REUABIUTY OF. AND INTERCORRELATIONS BETWEEN 
EACH CLASSROOM CLIML\TiE SCALE 



Scale 


No. of 
Items 


Oonbichot 
B|lUbUUy 


Scale InlercorreUtioni 








Div Sp 


Env Goal 


Sat Du 


Diff Comp Ind 


Diversity 


4 


0.50 


1.00' 0.00 


\.09 -0.05 


-0.1 1 0.03 


0.03 


^0.03 


-0.05 


Speed 


6 


0.66 


1.00 


0.35 -0.30 


-0.31 0.39 


0.3^ 


0.27 


-0.07 


Knvironmeni 


6 


0.63 




1.00 0.32 


0.26 -0.41 


-0.24 


-0.23 


0.04 


GtKil Direcliun 


7 


0.62 




1.00 


0.43-0.42 


-0.26 


-0.09 


0.07 


Satisfaction 


6 


0.80 






1.00 -0.48 


-0.34 


-0.05 


0.10 


DisorKunisution 


6 


0.66 






1.00 


0.33 


0.29 


0.11 


Uifricu!tV> 


5 


0.50 








1 .00 


0.23 


-0.02 


Competitiveness 


5 


0.53 










1.00 


0.12 


InHividualisation 
i. ■ • 


io 


0.71 












1.00 



SIGNlFICi^ JMT F VALUES FROM MULTIPLE REGRESSION ANALYSES FOR EACH 
CLIMATE SCALE AS CRITERION AND TREATMENT (ASEP/CONTROL). 
SESrIQ AND SEX AS PREDICTORS. 









F Values 






Criterion 


ASEP/control 


SES 






IQ 


Sex 


Diversity 




0.2 












Speed 




1.6 






L 


9.6* 




Environment 


A 


3.9* 












Goal Direction 




0.2 • ' 












Satisfaction 


A 


7.5** 












Disorganisation 




0.2 












Difficulty 




3.0 






L 


17.3*** 




Competitiveness 




2.9 






L 


7.4** 




Individ ul^sation 


' A 


I5.3*** 






L 


8.4** 





♦p<.05.**p<.OI. ***p<.OOI 

A Higher climate scores associated with ASEP 

L Higher climate scores associated with lower IQ 



The results of these analyses, which have been described in 
more detail in Fraser/ are displayed in the above table. This 
indicates that ASEP pupils, relative to control pupils, perceived 
their classes as being charaaerised by a significantly better " 
environment, significandy more satisfaction and significandy 
more individualisadon. The; table also indicates that pupils of 
high IQ,saw their classes as being characterised by less speed, less 
difficulty, less competitiveness and less individualisation than 
pupils of lo\yer IQ^in the same classrooms. • 

In interpreting these findings it should be realised that, 
because of the absence of pre-test climate data, it is possible that 
differences between the climate of ASEP and control classrooms < 
at the end of second term could be attributable in part tq 
differences ip climate existing at the start of the year. Therefore, 
some caution should be exercised in inteipreting these findings! 
Despite this caution, the present results are quite consistent with 
other findings from research related to ASEP. Firsdy, it was seen 
in Chapter 3 that ASEP pupils underwent more favourable 
changes in their enjoyment of science lessons during a year's 
science teaching than did control pupils. Secondly, Power and 
Tishei^ reported that the significant changes in climate scores 
experienced by ASEP classes over time were in the favourable 
direction for the large majority of climate dimensions. 



RELATIONSHIP BETWEEN LEARNING OUTCOMES 
AND CLASSROOM CLIMATE PERCEPTIONS 

This section will describe analyses in which climate scales cire 
employ^ as predictors of learning outcomes. These relationships 
wdre explored with the 53 1 pupils described in this chapter's first 
table, using scores on the same nine classroom climate scales and 
post-test scores on the seventeen outcomes employed in Chapter 
3. • ^ ' i 

The next table shows the simple correlation between each 
classroom climate scale ^d post-test performance on each of the 
seventeen learning outgome measures. This table indicates a 
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SimpleCofRiilioii 



i Crittfloii 


Div 


Sow 


Env 






Ditori 


TOES 1 (Library) 


0.13. ■ 


ft If 

•0.15 




■U.ll r 




412 , ■ 


toes : Index 


■ 0.2S**' ^ 


•0.08 


• 0.26*** 


415 : 


4,03 
0,01 


TOES 3 (Scalej) 


0.28"» 


410" , 


0.13' , 


. 410 


414- 


TOES 4 (Averagej) 


0.15 


412 


401 


427*** 


411 


406 


TOESS(M 


A Id* 


ft lAi 

•0.19' 


0.01 






Ml 


TOES 6 Graphi) 


0.21**' 


406 


0.13 


414" 


m- 
m 


408 

411- 


TOES 7 Comp) 


0.34*** 


414 


0.07 


404 


T0ES8(Expll) 


oj: 


403 


0.11 ■ . 


0.04 


m 


417*; 


T0ES9(Concl) 


0,2?*" 


An 

413 


A A1 






Mi 


TOUS P (Pniloi) 


U.iU 




U.UO 








TOUS H (HiiMoc) 


0.12 


410 


0.08 

0.1,? ■ 


418* 


414 


407 


TOUS N (Normal) 


0.22** 


423** 


405 


•401 


, Alt.S(Socinip) 


■0.05 ■ 


423**: 


0,26*** 


0,21** 


0.42*** 


431*** 


Alt. E (Enjoy) 


0.01 


430**« 


. 0,32*** 


0.27*** 


0.64*** 


435*** 


AH. I (Inltrest) 


405 


408 


0.17* 


0.32*** 


0.53*** 


431*** 


AtUo Enquiry; 


.' 0.08 N 


411 


O.00 


, 0.02 


0.10 ,, 




SMcAtts, 


0.14 


404 


410 


41 !■ 


402 


404 



428*t* 

418*1; 

^.18* .:: 
417*1: 
427***, 
424'*!: 



,417*; 

415 

410' 



415 
416* 

418* 
.424** 
425** 

413 
414 
0.11 

418* 



■0.18* 

:422** 

427*** 

413 

:430*** 

427**» 
416* 

:425**;: 

410 V 
0.01' 
0.19* 

0.13 • 

ao2 ' 



*p<.05:**p<.01;***ii<.00l 
. Lower climate scores associated with higlier criterion scores 
Saniple size was 1 53 class aptitude cells (20 sclibols) 
Critical r' 0.1 6 at .05 level of confidence ' 



412 
416*-- 

420* 

412 

419* 

■^423*^:^' 

434**Vv 

426'»*; 

424**r:: 

418* ' ? 
428**» • 

423**;;; 

0.05 : : ^ 
o.25*r- 

0.05 . 




reasonably strong rdationship overall betweejn classroom climate 
and learning outcome scoreSjtwith the median miagnitude of the 
correlation coefficient over all learning outcomes ranging from 
0.09 for Disorganisation to 0.19 for Individualisation. Further- 
more, the number of correlations significantly greater than zero 
at the 0.05 level of confidence was sixty- three out of 153, which is 
about eight times the number expeaed by chance. 

While this table provided information about simple correla- 
tions between learning outcomes and classroom climate, multiple 
regression analyses could be employed to investigate the relation- 
ship between classroom climate and changes in learning outcomes 
over the year, while controlling statistically for SES, IQ^ and sex. 
When such multiple regression analyses were^ carried out, 
nuinerous interesting relationships were found between individual 
climate dimensions and changes in individual learning outcomes. 
Taken together, the simple correlational analyses arid the multiple 
regression analyses provide strong evidence for the existence of a 
relationship between classroom climate and learning outcomes. 

This finding of a relationship between classroom climate 
and leaming^ outcomes is similar to the results from a series of 
studies reviewed b]^ Anderson and Walberg.^ . 

SUMMARY 

Like the previous chapter, this chapter described research 
involving one type of learning environment variables, namely 
classroom climate dimensions. These climate dimensions were 
employed both as criteria on which to compare ASEPand non- 
ASEP .classes and as predictors of Jeafning outcomes. 

When the classroom climate dimensions were employed as 
dependent variables, it was found that ASEP pupils held more 
favourable climate perceptions than non- ASEP pupils along the 
environment, satisfaction and individualisation dimensions. 

It was also found that IQ^mediated pupil climate perceptions: 
pupils of higher IQ^perceived their classes ash less speeded, less 
difficult, less competitive and less individualised than pupils of 
lower JQ^ in the same classrooms. When the classroom climate 
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variables were employed as independent variables, it was found 
that the relationship of classroom climate variables to leami[frg 
outcomes was relatively strong. - 

■ The findings from the two sets of analyses employing 
leaurning environment variables in this chapter, together ^Arith the 
results of research involving environmental variables from the 
previous chapter, can be taken to support the general usefulness 
of learning environment variables in curriculum evaluation re- 
search, y 
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APPENDIX I 



RESULTS FROM QUESTIOlsfNAIRE SURVEY OF SCIENCE TEACHERS IN 
MARCH AND NOVEMBER, 1974 







% response 
March 


% response 




A. Background Information 




/ X 

f y< 




(i) to (iii) Details of name, school, teaching exper- 
ience and qualifications 








. (iv) Present Knowledge of ASEP 








Nil or very hazy idea 


8 






A little knowledge but would need to know 
* more to consider using the materials 


14 


Not 




Sufficient knowledge to consider and use , 
:the materials 


: 69 


Applicable 




'. A great deal known about the ASEP approach 


10 ^ 






(v) Sources of Information about ASEP 


% useful or 
very useful 






. Journal articles (e.g. Lab Talk, A.S.T.J. etc.) 


68 






. ASEP units 


79 






ASEP teacher education materials ^ ' ^ 


49 






. ASEP newsletters - ^ 


48 
12 


Not 




— ^ — Newspaper articles 




Applicable 




Inservice courses 


49 








14 






— Participation as an -ASEP trials teacher 








Informal discussion among teachers within 
the schools 


1 

50 ' 






. ' Informal discussion among teachers from 
other schools 


40 






Pre-service courses (e.g. Dip. Ed) 


* 29 




; ■ . \ 


s 
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Details of Questionnaire 



%\etpofBe Mscb 



B. Form I Science in 1974 



(i) Approach to Form 1 Science — 

Form I'Scic'nce presented as a 
separate subject 

Form I Science integrated with 
other subjects (e.g. General 
Studies) 




Not 
Applicable 



Extemive 
Use 



(ii) Curriculum material intended for 
, use in 1974 at Form I level. 

. ASEP materials/ ! 

. J.S.S.P. materials \ ' 

' . ' Teacher prepared ' materials 

. Discoverj^in Science 



18 
26 



Use 



Exieflsnc 
Uae ^ 



54 
53 



32 
32 



( iii) Likely use.of ASEP matiprials 
in 1974 / 



Not/likely to be used 

Will obiain one or more units 

ff^T examination 
. /Will try at least one unit with 
/ class 

Have planned a program using 
j\S£E-matemls 



II • 

T19 

M 



26 
35 
41 

7 



United 



44 : 

40 
46 

21 



Sec actual use 
B(ii) above 



(iv) Method of making curriculum 
decisions for 1974 

Decision made by the teacher 
concerned 

Decision made by someone in 

authority (e.g. co-ordinator) 
. Decision made by science Starr 

after discussion ^ 
. A continuation of the previous 

Form 1 Science course 
. Other (Please Specify) 



IS 
8 
62 




7d 



c. 


fmroc Ideas ibout Fotm 1 Science 


March % rMpoiiM 


Novcm'Kef 


*« rrspomc 


D. 


frotat Knowlrdfe of ASEP* 


l^urm 1 Science ASKI* 
Atrc* l>M(r«« /^pe« DiMcrc* 


Kofm 1 
A|r«« 






A imm 1 S«Mni-« iitwrtc .... 








\ 










' 


Ai I'urai 4 -l«vcl AbKf malcfMb ■>« Jcittnci} lo . . 




















* i. pwptb tpvfii} mo*l i»r Ihcir iim« 
wurkNig in »m«U giuupi 


^> 


4 


Bl 


• 0 


A* 


I- 


08, 


0 


i 
1 
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APPENDIX II 



SAMPLE ITEMS FROM SCALES MEASURING LEARNING OUTCOMES 



Scale 


Scoring Procedure 


Sample Item 


TousP 


A score of 1 is given to the 
Keyed respoiise, D. 


When several new facts are discovcicd which 
do not fit a scientific theory, scientists are 
likely to 

A throw out the theory since the facts do 
not fit it 

D change the facts a littl^ so that they will 

fit the theory. 
C ignore the facts and keep the theory as 

it is 

D change the theory a little so that all 
facts will fit it. 


Tous H 


A score of 1 is given U. ihw 
keyed response. C. 


Scientitist study plants mainly to 
A help farmers to produce more food. 
B discover how to make new medicines. 
C understand how plants live and grow. 
D find out where plants will grow best. 


lousN 

\ 


A score of 1 is given to the 
keyed response, D. 


Bill always gets good results in school, likes 
to build model aeroplanes, and plays jokes 
on his classmatefs. . 

Frank gels high results in arithmetic, likes to 
read books, and plays baseball. Janet is 
serious and clever, and likes^'to dance. Who 
would become a scientist? 

A Bill only 

B Frank only * 

C Janet d^nly 

D Any one of the three 


Attitude 
to 

Inquiry 


Hach pupil allots 0-4 votes 
to each statement according 
to the extent to which a 
statement is like what 
he/she would have said. The 
item score is the number of 
votes given to No. I . 


What the surface of the moon is like 

.1. Robert said'he would borrow a tele- 
scope and study the moon. 

2. Pat said she would rather read a book 
about the moon. 

3. Dick said he would rather usk an ast* 
ronomer. 

4. Mary said she would rather ask her 
science teacher. 


Scien- 
tific 

Attitudes 


Iiach pupil picks which of / 
the pair of statements / 
belter describes him/her./ 
A score of 1 given to 
response it. / 


A. You show consideration for other people 

B. . You are observant. 
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APPENDIX III 



EXAMPLES OF STUDENT CHARTS SHOWIf^ PATTERNS' OF 
STUDENT ACTIVITY DURING THE AS£P UNIT 
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